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1.  Introduction 


1.1  Background 

Scalability  customarily  refers  to  the  ability  of  a  process  to  handle  increasing  amounts  of  work  or 
its  ability  to  be  readily  enlarged  (Bondi,  2000).  Scalability  can  also  be  referred  to  as  the  ability 
of  a  device  to  be  readily  reduced  in  size,  without  losing  functionality.  This  is  the  fourth  in  a 
series  of  experiments  designed  to  investigate  scalability  of  robotic  operator  controller  units 
(OCUs)  for  dismounted  Soldiers.  The  first  three  experiments  in  this  series  addressed  screen  size 
for  the  dismounted  Soldier’s  driving  camera  display  (Redden,  Pettitt,  Carstens,  and  Elliott, 
2008a),  controller  options  for  the  dismounted  Soldier  to  drive  the  robot  and  maneuver  the  robotic 
arm  (Pettitt,  Redden,  Carstens,  and  Elliott,  2008),  and  multimodal  displays  for  navigation 
(Redden,  Pettitt,  Carstens,  and  Elliott,  2008b).  The  scalability  component  of  interest  concerns 
how  best  to  accommodate  the  robotic  display  process  on  smaller,  lighter  OCUs  while 
maintaining  functionality,  adequate  visual  properties,  and  ease  of  use.  The  environments  of 
dismounted  Soldiers  are  rugged  and  physically  demanding,  and  Soldiers  must  carry  their  robotic 
OCUs  along  with  all  their  protective  and  fighting  equipment  in  these  challenging  environments. 
Relatively  large  displays  typically  used  in  a  stationary  environment  or  inside  a  combat  vehicle 
are  not  appropriate  and  could  have  an  adverse  impact  on  dismounted  Soldiers’  missions.  Thus, 
smaller  interfaces  are  needed  for  Soldiers  operating  in  the  dismounted  environment.  The  key  to 
successful  scaling  is  to  consider  the  range  of  devices  that  Soldiers  will  use  (e.g.,  vehicle-mounted 
robot  control  devices,  other  controller  devices)  and  also  their  context  of  use.  This  approach  to 
scalability  of  interfaces  ensures  that  training  transfer  is  easily  achieved  across  environments  and 
that  interfaces  can  be  tailored  to  the  environment  in  which  they  are  used. 

Two  findings  in  the  first  experiment  demonstrated  the  need  for  this  follow-on  experiment.  First, 
Redden  et  al.  (2008a)  found  a  few  areas  of  concern  regarding  the  use  of  monocular  head- 
mounted  displays  (HMDs)  for  robotic  teleoperation.  These  concerns  should  be  addressed  to 
ensure  that  perceptual  problems  such  as  binocular  rivalry  and  overstimulation  do  not  occur.  She 
and  her  colleagues  recommended  further  research  on  factors  having  a  potentially  adverse  impact 
on  helmet-  or  goggle-mounted  robotic  displays  if  these  displays  are  to  be  used  by  the  Army  for 
robotic  teleoperation.  Such  factors  include  interference  between  the  scenes  presented  to  two 
different  eyes,  potential  problems  with  prolonged  occlusion  of  one  eye,  the  inability  to  use 
compensatory  head  movements  to  center  the  display  on  the  visual  field,  and  the  effects  of  eye 
dominance  on  the  use  of  monocular  displays.  These  findings  are  consistent  with  the  literature  on 
HMDs. 

Second,  the  findings  from  the  Redden  et  al.  (2008a)  experiment  indicated  that  smaller  devices 
(i.e.,  3.5-in  diagonal  displays  and  6.5-in  diagonal  displays)  can  be  used  to  display  video 
sufficient  for  teleoperation  and  local  surveillance  with  small,  slow-speed  robots.  This  could 
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make  more  screen  space  available  for  other  objects  (if  the  total  display  size  is  larger  than  3.5  in) 
and  enable  the  use  of  lower  resolution  cameras,  which  would  allow  faster  video  processing  and 
lower  bandwidth  requirements.  This  finding  was  limited  to  users  driving  robots  using  a  camera 
display  while  seated  under  a  tent.  Thus,  additional  robotic  tasks  (e.g.,  navigation  using  map 
displays)  and  operating  environments  (e.g.,  dismounted  on-the-move)  need  to  be  investigated. 

1.2  Head-Mounted  Displays  (HMDs) 

HMDs  were  originally  developed  for  remote  control  of  robots  in  the  1960s  (Sutherland,  1968). 
They  have  continuously  evolved  since  the  1960s:  they  are  now  lighter  and  have  higher 
resolution,  and  some  are  so  small  they  can  be  clipped  onto  glasses.  They  are  used  everywhere 
from  industrial  plants  to  hospitals  to  dentist  offices.  The  original  concept  for  use  (robotics)  has 
received  more  emphasis  recently  and  their  use  for  defense-related  robots  is  being  championed. 
Those  who  are  enthusiastic  about  the  use  of  HMDs  argue  that  they  are  hands  free,  have  less  glare 
and  higher  resolution  than  a  handheld  display  (HHD),  and  are  extremely  lightweight.  They  also 
emphasize  that  the  HMDs  use  much  less  power  than  other  portable  displays  and  meet  the  light 
security  requirements  that  are  so  important  to  Soldiers.  Many  in  the  military  advocate  their  use 
during  dismounted  movement,  stating  that  HMDs  allow  Soldiers  to  observe  information 
presented  on  the  display  while  they  are  conducting  dynamic  operations.  While  these  are  very 
valid  points,  it  is  not  clear  if  HMDs  can  be  used  effectively  for  concentrated  monitoring  tasks 
during  operational  movement.  A  fundamental  question  is  whether  the  Soldiers  can  effectively 
divide  attention  between  their  surroundings  (e.g.,  terrain,  indicators  of  threat)  and  a  display  that 
can  partially  occlude  terrestrial  vision,  particularly  if  they  are  walking  over  uneven  terrain. 

HMDs  are  not  without  their  drawbacks.  The  following  perceptual  issues  are  representative  of 
those  found  in  the  literature  associated  with  the  use  of  HMDs  (Merhi,  Fauugloire,  Flanagan,  and 
Stoffregen,  2007;  Sheedy  and  Bergstrom,  2002;  Patterson,  Winterbottom,  and  Pierce,  2006): 

•  Brightness  and  contrast  perceptual  consequences  can  have  an  adverse  impact  on  vision. 

•  Both  real-world  and  HMD  imagery  must  be  within  the  user’s  depth  of  field,  dark  focus,  and 
dark  vergence. 

•  Accommodation-vergence  synergy  and  accommodation-vergence  mismatch  can  result  in 
blurred  objects  and  a  failure  to  detect  objects.  If  a  stimulus  is  blurred,  a  reflexive  change  in 
accommodation  attempts  to  resolve  the  blur.  The  knowledge  that  an  object  exists  that  is 
close  to  the  eye  may  cause  accommodation,  and  specific  conscious  mental  effort  can  result 
in  a  lapse  of  accommodation.  Accommodation  also  tends  to  change  if  there  is  a  change  in 
vergence.  This  mismatch  is  frequently  reported  with  HMD  use  and  can  create  eyestrain  or 
visual  discomfort. 

•  Restricted  field  of  view  can  impair  performance  and  adversely  affect  the  ability  to  see  the 
“whole  picture.” 
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•  Binocular  rivalry  is  a  state  of  competition  between  the  eyes  such  that  one  eye  inhibits  the 
visual  processing  of  the  other  eye.  The  views  delivered  to  the  two  eyes  can  be  very 
different  with  monocular  HMDs,  because  one  eye  sees  the  HMD  and  the  other  views  a 
real-world  scene. 

•  Eyestrain  and  blurry  vision  have  been  found  to  occur  significantly  more  with  HMDs  than 
with  other  displays. 

•  As  HMD  wearers  move  their  heads,  displayed  objects  in  front  continue  to  be  in  front  and 
this  is  unnatural.  Many  wearers  try  to  turn  their  heads  in  order  to  see  objects  outside  of  the 
camera’s  field  of  view. 

•  Motion  sickness  is  commonly  found  when  HMDs  are  worn  by  video  game  users  and  is 
often  explained  as  being  caused  by  sensory  conflict. 

Other  problems  with  HMDs  reported  in  the  literature  (Cobb,  1999;  Cobb  and  Nichols,  1998; 
Draper  et  al.,  2001;  Patterson  et  al.,  2006;  Elliott,  Duistermaat,  Redden,  and  van  Erp,  2007; 
Glumm  et  al.  1998;  Krupenia  and  Sanderson,  2007;  McCann,  Foyle,  and  Johnston  (1993); 
National  Research  Council,  1997;  Knight  and  Baber,  2007;  Turner  and  Carstens,  2007)  include 
the  following: 

•  Competition  for  the  attention  of  the  wearer/attentional  tunneling 

•  Interference  with  night  vision  devices 

•  Occlusion  of  full  field  of  view  if  the  HMD  is  not  see-through 

•  Fogging  due  to  weather 

•  Reduction  in  wearer  situation  awareness  (SA) 

•  Reduction  in  cross-country  movement  speeds 

•  Conflict  with  performance  of  other  critical  tasks 

•  Potential  for  placing  the  musculoskeletal  system  of  the  head  and  neck  under  increased 
levels  of  stress 

•  Known  to  be  nauseogenic.  This  problem  has  been  found  to  be  even  more  common  with 
postural  instability. 

•  Decreased  likelihood  of  detecting  unexpected  events  than  with  standard  displays 

The  National  Research  Council’s  Report  on  Tactical  Displays  for  Soldiers  (1997)  recommends 
that  if  the  display  occludes  the  Soldiers’  view,  then  HHDs  or  wrist-mounted  displays  should  be 
considered  as  an  alternative  to  HMDs,  because  the  loss  of  local  SA  may  be  unacceptable. 
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1.3  Handheld  Displays  (HHDs) 

Personal  Digital  Assistants  (PDA)  and  other  HHDs  are  becoming  mainstays  of  society.  It  is 
difficult  to  walk  down  the  street  without  seeing  someone  peering  at  their  PDA  or  BlackBerry*  as 
they  are  walking.  However,  most  individuals  who  attempt  to  walk  and  look  at  the  screen  of  their 
HHD  have  seriously  compromised  their  SA  of  their  surroundings.  Furthermore,  many  HHD 
screens  are  difficult  to  see  in  the  sun  because  of  glare.  Also,  they  are  difficult  to  read  due  to  low 
resolution,  small  size,  and  an  unstable  view  when  the  user  is  walking.  It  appears  the  most 
effective  way  to  use  HHDs  is  while  the  user  is  stationary.  During  stationary  operations,  it 
appears  that  HHDs  are  equally  effective  or  more  effective  than  the  HMDs.  For  example, 
Scribner,  Wiley,  Harper,  and  Kelley  (2007)  found  that  Soldiers  performed  better  in  terms  of 
enemy  target  hit  performance,  time  to  first  shot,  and  lower  enemy  engagement  rates  during 
defensive  shooting  (stationary)  with  a  forearm-mounted  small  display  than  with  an  HMD.  They 
did,  however,  state  that  the  HMD  appears  to  be  the  best  choice  for  a  single  task  when  hands-free 
convenience  is  imperative  but  not  when  dual-task  scenarios,  such  as  shooting  and 
communicating,  are  expected. 

1.4  Techniques  for  Dismounted  Soldier  Robotic  Operation 

A  recent  literature  search  on  robotic  operations  reveals  that  very  little  has  been  done  in  terms  of 
systematic  development  of  tactics,  techniques,  and  procedures  (TTPs)  for  robotic  operation  by 
dismounted  Soldiers.  The  Manned-Unmanned  Teaming  appendix  of  FM  3-90  (U.S.  Army, 

2001)  alludes  to  the  teaming  between  dismounted  Soldiers  and  unmanned  platforms.  It  identifies 
the  Small  Unmanned  Ground  Vehicle  (SUGV)  as  an  integral  part  of  the  fighting  team  and  states 
that  it  is  not  a  separate  platform  added  for  a  specific  mission.  It  is  a  highly  mobile  and 
reconfigurable  platform  intended  for  a  variety  of  missions.  It  further  states  that  manned  systems, 
teamed  with  unmanned  systems,  provide  the  Future  Combat  Systems  (FCS)-equipped  Brigade 
Combat  Team  Commander  with  continuous  and  wide-ranging  intelligence,  surveillance,  and 
reconnaissance  (ISR)  capability  necessary  to  accomplish  full-spectrum  operations.  Armed 
Robotic  Vehicles  (ARVs)  are  a  family  of  robotic  platforms  that  have  the  ability  to  provide  direct 
fire  for  offensive  operations  and  force  protection  in  addition  to  ISR  capabilities.  ARVs  support 
mounted  and  dismounted  operations  by  acting  as  combat  multipliers  by  providing 
reconnaissance,  surveillance,  and  target  acquisition;  additional  weapons  platforms;  and  limited 
communications  relays,  and  perhaps  most  importantly,  by  assuming  risks  that  Soldiers  would 
otherwise  have  to  assume.  An  example  of  a  mission  is  contained  in  the  appendix  of  FM  3-90.  In 
that  mission  a  dismounted  infantry  platoon  in  urban  operations  teams  with  ARVs  and  SUGVs  to 
clear  its  sector.  It  is  not  clear  whether  the  platoon  will  operate  the  robots  during  movement  or 
whether  the  robotic  operator  will  be  stationary  during  the  robotic  operation. 


BlackBerry  is  a  registered  trademark  of  Research  in  Motion  Limited. 
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A  major  focus  of  the  Air  Assault  Expeditionary  Force  (AAEF)  Spiral  D  Experiment  (2008)  was 
to  develop  TTPs  for  the  employment  of  a  network-enabled  small  unit  and  its  associated  battle 
command  and  robotics  systems.  The  experiment  was  built  around  an  Infantry  Brigade  Combat 
Team  (IBCT)  and  was  executed  by  a  small  unit  (Infantry  rifle  platoon  and  company 
headquarters)  enhanced  by  a  network  and  select  FCS  enablers.  The  TTPs  represent  insights 
gained  from  the  Experimentation  Force  (EXFOR),  technical  experts,  technology  training  teams, 
and  analysts.  The  technique  for  movement  discussed  in  that  document  centers  on  emplacement 
of  the  robot  by  the  Soldier  and  then  movement  of  the  Soldier  to  a  covered  and  concealed  position 
before  moving  the  robot.  Movement  of  the  robot  along  with  the  squad  was  discouraged, 
primarily  because  the  robots  used  in  the  experiment  violated  the  light  and  noise  security  of  the 
squad. 

1.5  Display  Type  and  Movement  Condition  Interaction 

The  advocates  of  HMDs  suggest  that,  under  certain  operating  conditions,  an  HMD  might  be  best 
because  it  provides  hands-free  operation  during  movement,  reduced  glare,  and  lighter  weight 
than  an  HHD  might.  However,  the  literature  suggests  that  during  stationary  operations,  an  HHD 
might  be  more  practical  because  of  all  the  perceptual  problems  associated  with  using  an  HMD. 
Whether  the  dismounted  Soldier  is  on  the  move  or  in  a  stationary  position  might  moderate  the 
effectiveness  of  the  two  display  modes.  Thus,  there  may  be  an  interaction  between  display  type 
and  movement  condition. 

1.6  Overview  of  Present  Research 

Our  study  investigated  the  effects  of  display  type  and  robotic  employment  techniques  on  robotic 
control  during  dynamic  dismounted  Soldier  operations.  It  took  place  at  Fort  Benning,  GA,  using 
Soldiers  from  the  Officer  Candidate  School  (OCS)  as  participants.  Two  types  of  employment 
techniques  and  two  types  of  displays  were  used.  The  employment  techniques  contrasted 
stationary  bounding  operation  with  operation  of  the  robot  while  the  Soldier  was  on  the  move. 

The  two  display  types  were  an  HHD  and  an  HMD.  After  training  on  the  operation  of  the  iRobot 
PackBot  system,  each  Soldier  completed  planned  exercises  four  times — once  with  each  display 
type  and  employment  technique  combination.  The  terrain,  targets,  and  hazards  were 
counterbalanced  along  with  the  display  and  movement  technique  type  to  control  for  order  effects 
(e.g.,  practice,  learning,  fatigue).  Effectiveness  of  display  type  and  employment  technique  was 
evaluated  based  on  objective  performance  data,  data  collector  observations,  and  Soldier 
questionnaires. 
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2.  Method 


2.1  Participants 

Thirty-six  Soldiers  from  the  OCS  participated  in  the  study.  Their  mean  age  was  26  years  and 
their  grades  ranged  from  E4  to  E8. 

2.2  Instruments,  Apparatus,  and  Techniques 
2.2.1  iRobot  PackBot  Robot 

The  iRobot  PackBot  Robot  is  a  portable  SUGV  reconnaissance  and  tactical  robot  that  can  enter 
and  secure  areas  that  are  either  inaccessible  or  too  dangerous  for  humans  (figure  1).  The 
PackBot  Explorer  Robot  payload  has  a  rotating  pan  and  tilt  head  equipped  with  multiple 
cameras. 


Figure  1.  PackBot  Explorer  Robot. 

2.2.2  Robotic  Vehicle  Displays 

Two  different  display  types  were  used  to  conduct  this  experiment.  Each  display  type  was 
plugged  into  the  existing  iRobot  PackBot  Robot  control  system  so  that  all  variables  except  the 
displays  remained  constant. 
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HHD 


The  HHD  was  a  PDA-sized  display,  as  shown  in  figure  2. 


Antenna 


Controls  and  Display 


Computational  Device 


Figure  2.  HHD. 

This  HHD  is  a  modified  prototype  Esterline-Mason  controller  that  integrates  the  control  inputs 
and  display  output.  The  display  is  a  6.5-in  diagonal  screen  with  1024  x  768  resolution.  The 
controller  was  modified  so  the  display  could  be  visible  in  full  sunlight  and  to  provide  controls 
since  the  integrated  controls  were  not  supported  by  the  software.  A  sunshield  was  placed  around 
the  display  and  a  modified  hand  controller  was  added  to  the  unit  (figure  3). 


Figure  3.  Modified  display  and  controls. 
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The  control  and  display  device  is  carried  by  the  operator.  The  computational  device  is  placed 
inside  a  standard  issue  day  assault  pack  (figure  4)  that  is  worn  by  the  operator.  The  antenna  is 
attached  to  the  outside  of  the  pack. 


Figure  4.  Assault  pack  and  F1F1D. 


HMD 

The  HMD  was  a  Land  Warrior-sized  display  (1.425  in,  800  x  600  resolution)  with  super  video 
graphics  array  (SVGA)  input.  The  system  consisted  of  a  modified  Tac-Eye  goggle-mounted 
display  from  Vuzix  and  a  standard  game  controller.  The  display  position  was  adjustable 
up/down  and  left/right,  and  each  participant  was  allowed  to  choose  the  eye  they  wanted  it  on  as 
well  as  its  actual  position.  The  only  modifications  to  this  device  were  that  the  goggle  lens  was 
removed  and  a  sunshield  was  added  to  reduce  glare  (figure  5). 


Figure  5.  HMD  with  sunshield. 
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The  computational  device  is  the  same  one  used  with  the  HHD  and  is  placed  inside  a  standard 
issue  day  assault  pack  that  is  worn  by  the  operator  (see  figures  2  and  4).  The  antenna  is  attached 
to  the  outside  of  the  pack. 

2.2.3  Questionnaires 

•  Demographic  Questionnaire.  The  demographic  questionnaire  (appendix  A)  was  developed 
to  collect  information  pertaining  to  Soldier  characteristics  and  experience,  especially  their 
knowledge  of  robotic  operation. 

•  Training  Questionnaire.  A  training  questionnaire  (appendix  B)  was  developed  to  collect 
information  concerning  the  adequacy  of  the  robotic  training,  the  display  training,  and  the 
training  course.  It  consisted  of  semantic  differential  rating  scales  and  open-ended 
questions. 

•  Post  Iteration  Questionnaire.  This  questionnaire  (appendix  C)  was  developed  specifically 
for  this  experiment  and  administered  during  this  evaluation.  The  post  iteration 
questionnaire  was  designed  to  be  administered  at  the  completion  of  each  trial  with  each 
system.  The  questionnaire  consisted  of  7-point  semantic  differential  scaled  questions  and 
open-ended  questions.  Questions  covered  Soldiers’  ratings  of  their  performance  and 
experiences  with  each  of  the  display  type  and  technique  combinations.  Open-ended 
questions  requested  the  Soldiers  to  write  their  evaluations  of  the  devices. 

•  Motion  Sickness  Assessment  Questionnaire  (MSAQ).  The  MSAQ  was  developed  by 
Gianaros  and  his  colleagues  (2001)  as  a  result  of  a  series  of  studies  illustrating  the 
dimensionality  of  motion  sickness  (appendix  D).  The  authors  used  exploratory  and 
confirmatory  factor  analyses  of  motion  sickness  descriptors  to  derive  and  verify  four 
dimensions  of  motion  sickness,  which  were  defined  as  gastrointestinal,  central,  peripheral, 
and  sopite-related.  These  dimensions  of  motion  sickness  were  then  used  to  construct  the 
MSAQ,  which  asks  the  participant  to  rate  phrases  concerning  how  they  feel  on  a  scale  of 

1  (not  at  all)  to  9  (severely). 

•  National  Aeronautics  and  Space  Administration  (NASA) -Task  Load  Index  (TLX).  The 
NASA-TLX  is  a  tool  that  allows  subjective  workload  assessments  by  operator(s)  working 
with  various  human-machine  systems  (Hart  and  Staveland,  1988).  It  uses  a  multi¬ 
dimensional  rating  procedure  that  derives  an  overall  workload  score  based  on  weighted 
averages  of  ratings  on  six  subscales.  These  subscales  include  mental  demands,  physical 
demands,  temporal  demands,  own  performance,  effort,  and  frustration.  It  can  be  used  to 
assess  workload  in  multiple  human-machine  environments.  The  version  of  the  NASA-TLX 
used  during  this  experiment  was  presented  to  the  Soldiers  on  a  computer.  Definitions  of 
each  scale  were  provided  on  laminated  paper  so  the  participants  could  refer  to  it  as  they 
were  providing  their  estimates  of  the  workload  associated  with  each  movement  condition 
on  the  various  scales. 
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•  End  of  Experiment  Questionnaire.  The  end  of  experiment  questionnaire  (appendix  E) 
required  the  Soldiers  to  rank  the  display  and  technique  combinations  and  to  answer  open 
ended  questions  about  the  displays  and  techniques. 

2.2.4  Dynamic  Robotic  Employment  Techniques 

Bounding  Technique 

In  the  bounding  technique,  the  robot  operator  is  stationary  during  robot  movement.  The  Soldier 
moves  to  the  transition  point  first  and  then  maneuvers  the  robot  past  his  position  to  the  next 
transition  point.  Then  the  operator  moves  up  to  the  position  of  the  robot.  The  operator  continues 
bounding  between  transition  points  until  reaching  the  end  of  the  course.  This  technique  requires 
the  use  of  the  camera  display  to  operate  the  robot  (figure  6). 


Figure  6.  Bounding  technique. 

Continuous  Movement 

In  the  continuous  movement  technique,  the  robot  operator  moves  in  conjunction  with  the  robot. 
The  robot’s  position  during  movement  is  to  the  rear  of  the  squad  formation.  At  the  first 
transition  point,  the  robot  is  called  forward  to  reconnoiter  areas  that  are  masked  to  line  of  sight. 
At  this  point,  the  operator  is  located  far  enough  behind  the  robot  so  that  he  must  use  the  camera 
display  to  maneuver  the  robot  rather  than  using  line  of  sight  to  the  robot.  Obstacles  and  other 
squad  members  block  the  operator  from  seeing  the  robot  directly  so  that  he  is  forced  to  use  the 
camera  display  to  maneuver  the  robot  (figure  7). 
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Figure  7.  Continuous  movement. 

2.2.5  Robotic  Course 

The  robotic  course  (figure  8)  was  ~200  m  in  length.  Both  the  robot  and  the  Soldier  teleoperating 
the  robot  negotiate  the  course.  The  course  is  designed  with  obstacles  that  mask  the  Soldier’s 
view  of  the  robot,  forcing  him  to  maneuver  the  robot  using  only  the  driving  camera  and  display. 
The  obstacles  included  a  tunnel,  hills,  a  covered  area,  and  tents.  The  course  path  was 
approximately  1  m  wide  and  was  clearly  marked  with  white  engineer  tape  on  the  left  and  right 
sides.  Target  silhouettes  and  simulated  improvised  explosive  devices  (IEDs)  that  could  be 
clearly  seen  through  the  driving  camera  were  placed  along  the  course  out  of  the  line  of  sight  of 
the  Soldier.  Three  transition  points  were  marked  with  red  flags.  The  transition  points  marked 
the  locations  where  the  Soldier  maneuvered  the  robot  from  a  location  behind  him  to  one  in  front 
of  him  in  order  to  reconnoiter  the  lane  for  the  target  silhouettes  and  simulated  IEDs. 
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Figure  8.  Robotic  course. 

2.3  Procedures 

2.3.1  Demographics 

The  Soldiers  reported  in  groups  of  four  for  one  day  each.  Upon  arrival,  they  were  briefed  on 
what  was  expected  of  them  and  how  the  information  they  provided  would  be  used.  All  of  the 
Soldiers  consented  to  participate  in  the  experiment.  Each  Soldier  then  completed  the 
demographic  questionnaire  (appendix  A)  and  was  given  a  roster  number. 

2.3.2  Training 

A  representative  of  iRobot  trained  the  Soldiers  to  teleoperate  the  robot  using  both  the  HHD  and 
HMD  display  while  conducting  both  the  continuous  movement  and  bounding  techniques.  They 
were  also  trained  on  the  features  of  the  robotic  driving  course  to  negate  the  effect  of  a  potential 
learning  curve.  After  the  training  session,  Soldiers  conducted  practice  field  trials  using  the  robot 
and  displays.  They  then  demonstrated  proficiency  with  the  system  by  performing  a  series  of 
maneuvers  for  the  trainer.  After  that,  the  Soldiers  completed  the  training  questionnaire 
(appendix  B). 

2.3.3  Experiment  Trials 

Soldiers  were  assigned  to  displays  and  techniques  according  to  the  matrix  presented  in  table  1 . 
The  matrix  was  constructed  using  all  possible  orders  of  movement  technique  and  display 
combinations,  as  listed: 

Condition  A  -  HHD/bounding  technique  combination 

Condition  B  -  HHD/operator  continuous  movement  technique  combination 
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Condition  C  -  HMD/bounding  technique  combination 

Condition  D  -  HMD/operator  continuous  movement  technique  combination 


This  Williams  square  design  controls  for  effects  due  to  order  and  also  for  effects  due  to 
preceding  device;  that  is,  B  does  not  always  follow  A  (Williams,  1949). 

The  Soldiers  were  told  that  they  were  acting  as  a  robotic  operator  and  maneuvering  through  the 
course  as  a  member  of  a  notional  Infantry  squad.  Their  primary  task  was  to  use  the  robot  to 
reconnoiter  areas  of  the  route  that  were  not  within  the  visual  range  of  the  squad  before  the  squad 
moved  through  the  area.  They  were  also  instructed  to  find  as  many  items  of  interest  (IEDs 
[figure  9]  and  silhouette  targets)  as  possible  and  move  through  the  course  as  quickly  as  possible. 
At  the  three  transition  points  on  the  course,  they  were  instructed  to  maneuver  the  robot  from  the 
rear  of  the  squad  formation  to  the  front  in  order  to  reconnoiter  areas  masked  by  obstacles.  A  data 
collector  acting  as  the  squad  leader  (figure  10)  recorded  the  number  of  items  that  were  found  and 
recorded  the  amount  of  time  the  Soldier  stopped  (during  the  continuous  movement  technique 
trials  only).  Another  data  collector  following  the  robot  recorded  the  driving  errors  (number  of 
times  the  robot  ran  over  or  into  an  object),  the  number  of  times  the  robot  went  off  course  (one 
track  drove  completely  outside  the  white  engineer  tape),  and  the  time  for  completion  of  the 
course.  The  robotic  course  was  negotiated  using  both  the  bounding  and  continuous  movement 
techniques. 

After  course  completion  with  each  display  type  and  technique  combination,  the  Soldiers 
completed  the  post  iteration  questionnaires  (appendix  C),  the  MSAQ  (appendix  D).  and  the 
NASA-TLX  and  waited  until  the  course  was  ready  for  the  next  iteration. 


Figure  9.  IED  on  course.  (Note:  The  robot  is  off  course.) 
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Figure  10.  Data  collectors. 

2.3.4  Post  Experiment  Questionnaire 

Upon  completion  of  all  trials  with  each  display  and  technique  combination,  the  Soldiers  were 
given  an  end  of  experiment  questionnaire  (appendix  E)  designed  for  this  experiment  to  assess  the 
degree  of  acceptance  for  each  combination  and  their  overall  preferences. 


3.  Results 


3.1  Demographics 

The  36  OCS  Soldiers  who  participated  in  this  experiment  averaged  4  years  in  the  military.  Some 
of  the  subjects  were  prior  service  enlisted  from  a  variety  of  Army  fields,  including  Infantry, 
Combat  Engineer,  Intelligence  Analyst,  and  Special  Electronic  Devices  Repairer.  Other  subjects 
were  recent  basic  training  graduates  that  enlisted  under  the  college  option  for  OCS.  Four  of  the 
Soldiers  were  left-handed  and  14  wore  prescription  lenses.  Twenty-eight  were  right  eye 
dominant.  Detailed  results  from  the  demographic  questionnaire  can  be  found  in  appendix  A. 

3.2  Training 

The  participants  were  generally  positive  about  the  quality  and  quantity  of  training  on  the  use  of 
the  robot,  movement  techniques,  and  displays.  Soldiers  also  rated  the  practice  lane  highly.  They 
indicated  that  learning  to  drive  the  robot  with  the  HMD  was  somewhat  harder  than  learning  to 
drive  the  robot  with  the  HHD,  but  the  ratings  for  the  training  they  received  on  both  displays  were 
high.  Detailed  comments  on  the  training  questionnaire  can  be  found  in  appendix  B. 
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3.3  Experiment  Trials 
3.3.1  Performance  Data 

One  participant  was  deleted  from  the  analyses  reported  in  this  section;  this  Soldier  was  an 
extreme  outlier  on  all  dependent  measures.  Thus,  the  sample  size  used  in  the  analyses  in  this 
section  is  35. 

Course  Completion  Times 

The  mean  course  completion  times  for  each  experimental  condition  are  shown  in  table  2  and 
figure  11.  A  2  x  2  repeated  measures  analysis  of  variance  (ANOVA)  indicates  that  there  was  a 
significant  main  effect  for  display  [F(l,34  )  =  1 1.5 ,p  =  0.002,  r\2p  =  0.253],  The  main  effect  for 
movement  was  not  statistically  significant  [F(l,34)  =  1.74 ,p  =  .196,  r\2p  =  0.049],  nor  was  the 
display  x  movement  interaction  [F(l,34)  <  1.00]. 

Table  2.  Mean  course  completion  times  by  condition. 


Condition 

Display 

Technique 

Mean 

(S) 

SD 

(s) 

A 

HHD 

Bounding 

218 

84 

B 

HHD 

Continuous 

240 

117 

C 

HMD 

Bounding 

254 

107 

D 

HMD 

Continuous 

288 

168 

Figure  11.  Mean  course  completion  times  (seconds). 
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Driving  and  Off  Course  Errors 

The  mean  number  of  driving  errors  for  each  condition  is  shown  in  table  3  and  figure  12.  A  2  x  2 
ANOVA  indicates  that  there  was  a  significant  main  effect  for  display  [F(l ,34  )  =  5.00,/?  =  0.032, 
r|2p  =  0.128]  and  for  movement  [F(l,34  )  =  4.63, p  =  0.039,  ri2p  =  0.120].  The  display  x 
movement  interaction  was  not  statistically  significant  [F  <  1.00]. 

Table  3.  Mean  driving  errors. 


Condition 

Display 

Technique 

Mean 

SD 

A 

HHD 

Bounding 

1.89 

2.61 

B 

HHD 

Continuous 

2.71 

2.97 

C 

HMD 

Bounding 

3.00 

3.06 

D 

HMD 

Continuous 

3.74 

3.88 

Figure  12.  Mean  driving  errors. 


Table  4  and  figure  13  show  the  mean  number  of  times  the  robot  was  driven  off  course  in  each  of 
the  display-movement  conditions.  Both  of  the  main  effects  for  display  [F(l,34  )  =  15.6,  p  < 
0.001,  r|2p  =  0.315]  and  movement  [F(l,34  )  =  4.30, p  =  .046,  r|2p  =  0.112]  were  statistically 
significant.  The  display  x  movement  interaction  was  not  statistically  significant  [F  <  1.00]. 

Table  4.  Mean  number  of  times  off  course. 


Condition 

Display 

Technique 

Mean 

SD 

A 

HHD 

Bounding 

2.14 

2.43 

B 

HHD 

Continuous 

3.31 

3.52 

c 

HMD 

Bounding 

4.66 

4.53 

D 

HMD 

Continuous 

5.34 

3.61 
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Figure  13.  Mean  number  of  times  off  course. 

Robotic  Target  Detection 

The  means  for  the  target  detection  measure  (number  of  items  and  enemy  personnel  detected)  are 
shown  in  table  5  and  figure  14.  There  was  no  significant  main  effect  for  display  [F  <  1.00].  The 
main  effect  for  movement  was  statistically  significant  [F(l,34  )  =  4.43 ,p  =  0.043,  r\  p  =  0.1 15]. 
The  display  x  movement  interaction  was  not  significant  [F  <  1.00]. 


Table  5.  Mean  number  of  items  detected. 


Condition 

Display 

Technique 

Mean 

SD 

A 

HHD 

Bounding 

1.63 

1.19 

B 

HHD 

Continuous 

1.14 

1.06 

C 

HMD 

Bounding 

1.54 

1.09 

D 

HMD 

Continuous 

1.31 

1.25 
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Figure  14.  Mean  number  of  items  detected. 

Stops  During  Continuous  Movement 

Two  of  the  dependent  variables,  number  of  times  that  the  robot  stopped  and  total  time  stopped, 
apply  only  to  the  continuous  movement  conditions.  These  measurements  were  an  indication  of 
the  effect  the  display  configuration  had  on  the  ability  of  the  Soldier  to  continue  movement  while 
using  the  display.  More  frequent  stops  and  longer  stops  indicate  that  the  display  configuration  is 
adversely  affecting  the  ability  of  the  Soldier  to  continue  movement.  The  mean  number  of  times 
stopped  for  the  two  display  conditions  is  shown  in  table  6.  A  paired  samples  t-test  indicates  that 
there  was  no  significant  difference  between  the  means:  t(df=34)  =  0. 1 l,p  =  0.864,  d  =  0.04. 
Note:  Cohen’s  d  is  an  index  of  effect  size,  the  difference  between  the  means  divided  by  the 
pooled  standard  deviation. 

Table  6.  Mean  number  of  times  stopped. 


Display 

Mean 

SD 

HHD 

4.94 

5.27 

HMD 

5.11 

4.29 

Table  7  shows  the  average  time  stopped  with  the  two  display  conditions.  This  time  is  an 
accumulation  of  the  running  times  from  each  stop.  There  was  no  significant  difference  between 
the  means:  t(df=34)  =  0.99 ,p  =  0.331,  d  =  0.25. 


Table  7.  Total  time  stopped. 


Display 

Mean 

(S) 

SD 

(s) 

HHD 

28.3 

51.4 

HMD 

41.8 

76.2 
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The  distribution  of  amount  of  time  stopped  on  the  course  was  highly  skewed;  most  participants 
stopped  for  30  s  or  less,  but  a  few  stopped  for  several  minutes.  The  time  stopped  data  were 
transformed  to  logarithms  to  obtain  a  better  approximation  of  normality  (Winer,  Brown,  and 
Michels,  1991).  There  was  no  significant  difference  between  the  two  display  conditions  for  the 
transformed  data  ( t  <  1.00). 

Soldiers  had  to  stop  frequently  and  for  long  periods  of  time  with  both  display  configurations.  In 
effect,  many  of  the  Soldiers  were  unwittingly  turning  the  continuous  movement  into  a  bounding 
movement  because  they  were  unable  to  teleoperate  the  robot  while  walking. 

3.3.2  Subjective  Soldier  Data 

Post  iteration  Questionnaire  Results 

Soldiers  rated  their  abilities  to  maintain  SA  of  their  surroundings  highest  with  the  HHD  and 
bounding  movement  and  lowest  with  the  HMD  and  continuous  movement.  One  Soldier 
commented  that  with  the  HHD,  he  was  able  to  look  around  at  his  surroundings  while  driving  the 
robot.  Another  commented  that  the  HMD  impeded  his  SA  during  continuous  movement  and 
blocked  him  from  seeing  objects  on  the  ground.  Several  indicated  that  they  were  more  aware  of 
their  surroundings  when  they  were  moving  with  the  bounding  technique  because  they  could 
concentrate  on  their  immediate  area  as  they  moved  to  the  next  transition  point. 

Soldiers  rated  their  abilities  to  move  tactically  the  highest  with  the  HHD  and  bounding  technique 
and  the  lowest  with  the  HMD  and  continuous  movement  technique.  Many  Soldiers  commented 
that  it  was  easier  to  maneuver  the  robot  while  they  were  stationary  than  while  they  were  moving. 
One  Soldier  commented  that  he  had  to  close  one  eye  to  use  the  HMD  and  that  this  made  the 
continuous  movement  impossible. 

There  were  a  total  of  23  complaints  of  eyestrain  reported  with  HMD  use  and  only  4  with  HHD 
use.  There  were  also  twice  as  many  complaints  of  disorientation  with  HMD  use  as  there  were 
with  HHD  use.  Soldiers  complained  of  competition  between  their  eyes  when  the  HMD  was 
worn  and  rated  the  comfort  of  using  the  HMD  lower  than  they  did  that  of  the  HHD.  Despite  the 
modified  sunshields  on  both  displays,  there  were  still  several  complaints  concerning  glare  with 
both  displays.  However,  there  were  more  complaints  with  the  HHD  (12  complaints  of  glare  with 
the  HHD  on  the  post  iteration  questionnaire  versus  3  complaints  with  the  HMD).  Ratings  of  the 
effect  of  glare  on  the  displays  were  3.59  for  the  HHD  and  4.13  for  the  HMD. 

The  Soldiers  who  wore  eyeglasses  were  unable  to  wear  their  glasses  while  using  the  HMD. 

There  were  also  numerous  comments  when  the  HMD  was  worn  concerning  fogging,  competition 
between  the  two  eyes,  and  display  movement  causing  Soldiers  to  have  to  hold  the  display  steady 
with  one  hand. 

Detailed  results  from  the  post  iteration  questionnaire  can  be  found  in  appendix  C. 
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Motion  Sickness  Assessment  Questionnaire 

Results  of  the  MSAQ  indicate  that  the  experiment  Soldiers  experienced  very  little,  if  any,  of  the 
symptoms  of  motion  sickness  during  this  experiment.  The  MSAQ  ratings  are  shown  in 
appendix  D. 

NASA-TLX 

Thirty- three  of  the  35  participants  completed  the  NASA-TLX  inventory  after  each  of  the  four 
experimental  iterations. 

Table  8  and  figure  15  show  the  total  workload  means  for  the  NASA-TLX  for  the  conditions. 
The  main  effect  for  display  was  statistically  significant  [F(l,32  )  =  20.2, p  <  0.001,  r|2p  =  0.387], 
The  movement  main  effect  was  not  statistically  significant  [F(l,323  )  =  2>.\6,p  =  0.085,  r|  p  = 
0.090].  The  display  x  movement  interaction  was  not  statistically  significant  [F)  <  1.00]. 

Table  8.  NASA-TLX  total  workload  means. 


Condition 

Mean 

SD 

A.  HHD/Bounding 

39.1 

16.3 

B.  HHD/Continuous 

41.2 

18.2 

C.  HMD/Bounding 

45.4 

21.1 

D.  HMD/Continuous 

49.9 

19.8 

Figure  15.  NASA-TLX  mean  workload  scores. 

The  means  for  the  NASA-TLX  subscales  for  each  display  type  are  shown  in  figure  16  and 
table  9.  With  the  exception  of  the  Physical  scale,  the  HMD  had  a  higher  mean  rating  than  the 
HHD  on  all  the  scales. 
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Table  9.  NASA-TLX  scale  means. 


Scale 

HHD 

HMD 

Mean 

SD 

Mean 

SD 

Mental 

9.74 

7.46 

11.19 

7.75 

Physical 

3.20 

2.98 

2.94 

3.28 

Temporal 

7.11 

5.95 

8.43 

6.79 

Performance 

8.01 

5.72 

8.87 

4.79 

Effort 

8.94 

6.48 

9.91 

6.00 

Frustration 

3.72 

6.32 

6.40 

7.25 

Figure  16.  NASA  TLX  subscale  means. 

End  of  Experiment  Questionnaire 

Soldiers  preferred  the  HHD  to  the  HMD  and  the  bounding  technique  to  the  continuous 
movement  technique  for  the  same  reasons  they  listed  in  the  post  iteration  questionnaires. 

Soldiers  suggested  several  improvements  for  the  displays.  They  suggested  a  better  system  for 
preventing  glare  or  washout  with  the  HHD.  Several  suggested  adding  a  better  sunshield  and  two 
suggested  that  instead  of  a  display  that  is  held  flat  when  using  the  controls,  it  would  be  better  to 
have  one  that  tilts  or  one  that  is  placed  at  a  45°  angle  so  the  sun  does  not  shine  directly  on  it.  For 
the  HMD,  they  suggested  a  better  mounting  system  was  needed.  Problems  were  experienced 
with  instability,  lack  of  compatibility  with  glasses,  and  with  distance  from  the  eye. 

Detailed  results  from  the  end  of  experiment  questionnaire  can  be  found  in  appendix  C. 
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4.  Discussion 


4.1  HHD 

The  experiment  results  pertaining  to  the  issues  associated  with  the  use  of  HHDs  in  the  literature 
are  as  follows: 

•  Several  Soldiers  commented  that  during  continuous  movement,  the  requirement  to  look  at 
the  screen  of  their  HHD  seriously  compromised  their  SA  of  their  surroundings. 

•  The  screen  of  the  HHD  was  difficult  to  see  in  the  sun  because  of  glare  (even  after  the 
sunshield  was  added). 

•  There  were  only  a  couple  of  negative  comments  concerning  the  resolution  and  small  size  of 
the  HHD. 

•  Concern  was  expressed  about  the  fact  that  the  HHD  required  the  use  of  the  hands. 

The  findings  of  this  experiment  supported  the  findings  of  Redden  et  al.  (2008a)  that  smaller 
display  sizes  (in  this  case,  a  display  with  a  6.5-in  diagonal)  are  sufficient  for  teleoperation,  which 
includes  detection  of  objects  within  the  path  of  the  robot  that  may  impede  movement.  It  is 
important  to  note  that  this  finding  pertains  to  the  display  for  the  driving  camera  only  and  is  not 
generalizable  to  displays  for  other  camera  types.  For  example,  wider  fields  of  view  and 
magnification  may  be  needed  for  other  missions,  such  as  a  scout  mission  when  the  objective  is  to 
search  for  targets  outside  the  path  of  the  robot.  This  could  drive  the  requirement  for  higher 
resolution  and  a  larger  screen  size  for  display  of  that  camera’s  picture  and  this  issue  needs  to  be 
explored. 

4.2  HMD 

The  experiment  results  pertaining  to  the  perceptual  issues  associated  with  the  used  of  HMDs  in 
the  literature  are  as  follows: 

•  Brightness  and  contrast  were  a  problem  when  the  HMD  was  used  in  the  bright  sunlight. 

The  addition  of  an  “eyecup”  helped,  but  did  not  completely  alleviate  the  problem. 

•  Accommodation- vergence  synergy  and  accommodation-vergence  mismatch  were  not 
reported  as  such.  However,  since  this  mismatch  is  frequently  reported  with  HMD  use  and 
can  create  eyestrain  or  visual  discomfort,  it  may  have  been  the  reason  for  the  high  number 
of  reported  eyestrain  incidences. 
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•  The  restricted  field  of  view  created  by  wearing  the  HMD  over  one  eye  impaired  walking 
performance  during  continuous  movement  and  the  Soldiers’  ability  to  see  objects  in  their 
environments. 

•  Binocular  rivalry  occurred  and  created  competition  between  the  eyes  such  that  one  eye 
inhibited  the  visual  processing  of  the  other  eye.  The  views  delivered  to  the  two  eyes  were 
very  different  with  the  monocular  HMD  and  many  Soldiers  reported  that  they  had  to  close 
the  other  eye  in  order  to  avoid  losing  focus. 

•  Eyestrain  and  blurry  vision  were  a  problem  and  occurred  considerably  more  often  with  the 
HMD  than  with  the  HHD. 

•  As  HMD  wearers  moved  their  heads,  displayed  objects  in  front  continued  to  be  in  front  and 
this  was  unnatural.  Many  wearers  tried  to  turn  their  heads  to  see  objects  outside  of  the 
camera’s  field  of  view. 

•  Motion  sickness  was  not  reported  as  a  problem  with  the  HMD  during  this  experiment. 

Other  problems  reported  with  the  HMD  included  the  following: 

•  Occlusion  of  the  full  field  of  view 

•  Fogging  due  to  weather 

•  Reduction  in  wearer  SA 

•  Reduction  in  movement  speeds 

•  Instability  of  the  HMD  mount 

•  Incompatibility  of  the  HMD  with  eyeglasses 

•  Inability  to  place  the  HMD  at  the  desired  distance  from  the  eye 

Soldiers  performed  significantly  worse  with  the  HMD  than  they  did  with  the  HHD  on  course 
completion  times,  driving  errors,  and  the  number  of  times  they  drove  off  course.  The  Soldiers 
also  preferred  the  HHD  to  the  HMD  and  rated  the  workload  with  the  HHD  lower.  These 
findings  are  similar  to  the  findings  of  Redden  et  al.  (2008a).  Redden  and  her  colleagues  found 
that  Soldiers  completed  a  leg  of  their  driving  course  significantly  slower  with  the  HMD  than  they 
did  with  the  PDA-sized  HHD  and  there  was  a  trend  for  the  Soldiers  to  perform  worse  with  the 
HMD  on  every  performance  measure. 

4.3  Bounding  Movement  Technique 

The  bounding  movement  technique  was  the  preferred  movement  technique  and  Soldiers 
performed  best  with  this  technique.  Soldiers  reported  that  they  were  able  to  teleoperate  the  robot 
more  efficiently  when  they  were  stationary  and  that  they  were  also  able  to  attend  to  the 
environment  and  find  targets  more  efficiently  when  they  were  walking  and  not  having  to 
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teleoperate  the  robot.  Thus,  their  SA  of  the  robot  and  their  SA  of  their  surroundings  was  better 
when  the  bounding  movement  technique  was  used. 

4.4  Continuous  Movement  Technique 

Soldiers  reported  that  the  HMD  blocked  their  view  of  their  surroundings  while  moving  and 
trying  to  teleoperate  the  robot.  They  also  overwhelmingly  felt  that  the  continuous  movement 
technique  with  either  display  reduced  their  efficiency  of  movement  and  their  SA  of  their 
surroundings.  Movement  during  robotic  control  is  somewhat  analogous  to  control  of  multiple 
robots  and  appears  to  require  at  least  a  semi-autonomous  robot  before  it  will  be  a  realistic 
technique  option.  Performance  degradation  during  the  continuous  movement  technique 
generally  came  from  two  sources:  event  time-line  conflicts  and  attentional  limitations. 
Simultaneous  conflicts  between  requirements  for  focused  attention  frequently  arose  as  Soldiers 
negotiated  obstacles  themselves  while  trying  to  negotiate  other  obstacles  with  the  robot. 
Teleoperation  alone  requires  a  great  deal  of  operator  attention  because  remote  perception  is 
essential  and  extremely  challenging,  creating  such  problems  as  scale  ambiguity,  viewpoint 
mismatch,  and  lack  of  motion  feedback.  (Chen,  Haas,  Pillalamarri,  and  Jacobson,  2006).  Other 
problems  created  by  teleoperation  include  limited  view,  degraded  depth  perception,  degraded 
video  image,  and  time  delay  (latency)  (Chen  et  al.,  2006).  The  camera  latencies  experienced  in 
this  experiment  were  a  continuous  source  of  frustration  to  the  experiment  Soldiers  and  were 
attributed  by  the  Soldiers  to  several  driving  errors  and  slow  task  performance.  While  many  of 
these  problems  were  present  when  Soldiers  operated  the  robot  using  the  bounding  movement 
technique,  they  appeared  to  affect  performance  more  severely  during  continuous  movement 
because  attention  was  more  limited  and  workload  was  higher. 


5.  Conclusions 


The  Soldiers  performed  better  with  the  HHD  than  they  did  with  the  HMD  used  in  this 
experiment.  Their  course  completion  times,  driving  errors,  and  the  number  of  times  they  drove 
off  course  were  all  lower  with  the  HHD.  The  Soldiers  also  preferred  the  HHD  to  the  HMD  and 
rated  the  workload  with  the  HHD  lower. 

Soldiers  preferred  the  bounding  technique  to  the  continuous  movement  technique.  Fewer  driving 
and  off  course  errors  were  made  and  more  items  were  detected  with  the  bounding  technique  than 
with  the  continuous  movement  technique.  Until  robots  become  more  autonomous  in  their 
navigation,  robotic  control  during  Soldier  movement  is  beyond  the  multitasking  ability  of  most 
Soldiers.  As  reliable  autonomy  in  robots  increases  and  allows  the  robot  to  continue  productive 
movement  without  human  intervention,  Soldiers  should  have  more  available  free  time  to  divert 
their  attention  away  from  the  robot  and  toward  their  route  of  movement  and  their  own 
environment,  enabling  side-by-side  movement. 
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6.  Recommendations 


Dixon,  Wickens,  and  Chang  (2003)  indicate  that  teleoperation  creates  a  higher  workload  for  the 
robotic  operator  than  operation  of  autonomous  robots.  If  simultaneous  and  close  movement  of 
robots  with  Soldiers  is  desired,  it  appears  robots  will  require  at  least  some  level  of  autonomy. 
However,  even  with  autonomous  operation,  workload  reduction  is  dependent  upon  the  reliability 
of  the  autonomous  system  (Dixon  and  Wickens,  2004). 
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Appendix  A.  Demographics 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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APPENDIX  A 


DEMOGRAPHICS 
SAMPLE  SIZE  =  36 


MOS  RANK 


09S 

-  7 

88H 

-  1 

E2 

-  2 

1 1B 

-  2 

92Y 

-  1 

E3 

-  5 

21B 

-  10 

96  B 

-  1 

E4 

-  18 

21C 

-  9 

OCS 

-  1 

E5 

-  8 

35F 

-  1 

NR 

-  3 

E6 

-  1 

Officer  -  1 

NR  -  1 


1 .  What  is  your  age?  26  years  (mean  )  -  19-37  range 

2.  With  which  hand  do  you  most  often  write?  Right -30  Left -4  NR -2 

3.  With  which  hand  do  you  most  often  fire  a  weapon?  Right -31  Left -4  NR-1 

4.  Do  you  wear  prescription  lenses?  Yes  -  14  No  -  22 

5.  If  Yes,  which  do  you  most  often  wear?  Glasses  - 10  Contacts  -  3 

6.  Which  is  your  dominant  eye?  Right -28  Left -8 

7.  How  many  months  in  military?  45  (mean) 

8.  Using  the  scale  below,  please  rate  your  skill  level  for  each  of  the  following  activities. 


None 

Beqinner 

Intermediate 

Expert 

1 

2 

3 

4 

MEAN  RESPONSE 

Land  navigation 

2.71 

Operating  ground  unmanned  vehicles 

1.94 

Operating  unmanned  aerial  vehicles 

1.29 

Target  detection  and  identification 

2.26 
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Appendix  B.  Training 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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APPENDIX  B 


TRAINING 
SAMPLE  SIZE  =  36 


1.  Using  the  scale  below,  please  rate  the  training  that  you  received  in  the  following 
areas: 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN  RESPONSE 

How  to  drive  the  robot 

5.69 

Time  provided  to  practice  driving  the  robot 

5.18 

How  to  negotiate  the  robotic  course 

5.29 

How  to  complete  the  movement  to  the  objective 

5.51 

How  to  use  the  HMD 

5.29 

How  to  use  the  HHD 

5.62 

Evaluation  of  the  practice  lane 

5.29 

How  to  employ  the  bounding  movement  technique 

5.54 

How  to  employ  the  continuous  movement  technique 

5.49 

How  well  you  expect  to  perform  on  the  actual  course 

5.06 

Overall  evaluation  of  the  training  course 

5.71 

2.  What  were  the  easiest  training  tasks  to  learn? 


Comments  Responses 

Movement  techniques.  2 

Bounding  movements  with  the  iRobot.  1 

Bounding  with  the  handheld  display.  2 

Bounding.  These  were  two  separate  tasks.  Running  to  a  point,  then  driving  1 
the  robot  were  the  easiest  to  learn  and  perform  adequately. 

Using  the  handheld  controllers  to  operate  the  robot  2 

How  to  control  the  robot.  3 

Simple  robot  movements  while  remaining  stationary.  2 

I  think  that  the  easiest  things  to  learn  were  to  drive  while  standing  still.  I  think  it  1 
was  easier  to  learn  with  the  handheld  than  the  mounted. 

The  basic  controls  and  use  of  the  HHD.  1 

Driving  the  robot  with  the  HHD.  7 

Learning  how  to  use  the  HHD.  2 

Learning  the  speed  of  the  robot.  1 

Moving  the  robot  left/right.  1 

Forward  and  reverse.  3 
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Comments 


Responses 


Handling  the  robot  at  higher  speeds  via  the  display  1 

Identifying  obstacles  and  objects  in  the  path  of  the  robot.  1 

I  really  felt  as  though  with  the  training  and  hands-on  provided  that  all  tasks  1 
were  very  easy  to  learn. 

3.  What  were  the  hardest  training  tasks  to  learn? 

If  I  had  to  pick  out  a  difficulty,  I  would  have  to  say  that  the  lag  time  in  the  1 

display  makes  the  tasks  harder;  however  I’m  really  reaching  for  a  negative 
there  because  the  tasks  were  very  easy. 

Adapting  to  the  camera  and  robot  delay  and  maneuvering  the  robot  via  1 

joystick  was  a  bit  difficult. 

It  was  difficult  to  learn  the  HMD  display.  This  was  largely  due  to  a  connection  1 
problem  which  was  slow  for  download  of  feed  as  well  as  obtaining  the 
movement  of  the  robot. 

Moving  the  iRobot  while  continuously  moving.  1 

Moving  continuously  with  the  HMD.  3 

The  HMD  moving  because  you  had  to  hold  the  eyepiece  and  control  it.  1 

Using  the  HMD  to  view  where  you  are  going  with  the  robot  because  of  the  1 

difficulty  focusing  on  the  display  while  you  are  moving  and  trying  to  watch 
where  you  are  going. 

Bounding  with  the  HMD.  1 

Using  the  HMD  -  all  tasks.  5 

Driving  the  robot  with  the  HMD.  3 

I  found  that  it  was  harder  to  learn  with  the  HMD  versus  the  HHD  and  it  was  1 
also  more  difficult  to  not  look  at  the  robot  when  using  the  HMD  display. 

Learning  to  drive  the  HMD  while  keeping  both  eyes  open.  1 

Manipulating  the  HMD  and  the  controls  at  the  same  time.  1 

Complex  turns  and  obstacle  negotiation  along  with  target  identification  while  1 
moving. 

Maneuvering  the  robot,  particularly  using  the  head  gear.  1 

Moving  the  robot  in  reverse  and  right/left  reverse.  1 

Being  able  to  back  up  and  turn  when  you  run  into  something.  1 

Having  the  robot  make  tight  turns.  1 

Turning  the  robot.  1 

Turning  and  walking  while  operating  the  robot.  1 

Navigating  from  covered  to  sun  exposed  areas  due  to  glare.  2 

Navigating  the  robot  through  narrow  corridors.  1 

Some  of  the  driving.  1 

What  was  on  the  screen  was  closer  as  it  seems.  1 
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4.  What  are  your  comments  on  the  training  course? 


Comments  Responses 

Great  training  course.  2 

It  was  a  good  challenging  course.  1 

The  training  was  awesome;  it’s  good  to  work  with  new  things.  1 

It  was  fun!!!  2 

The  training  was  excellent,  but  I  think  that  you  should  factor  in  the  dominant  1 
hand  in  addition  to  the  eye 

A  device  that  has  been  manufactured,  rather  than  rigged,  might  assist  with  the  1 
use  of  the  HMD. 

A  little  more  time  to  practice  with  the  robots  would  have  been  helpful  before  1 
hitting  the  actual  course  or  perhaps  a  practice  run  on  the  course  before 
taking  the  test. 

Good  course  but  the  trail  was  kind  of  difficult  to  decipher.  1 

I  think  personnel  should  have  a  little  longer  duration  to  learn  how  to  drive  the  2 
robot. 

Incorporate  objects  to  find  so  user  can  have  examples  of  what  to  look  for  or  so  1 
user  can  get  trained  on  what  the  objects  will  look  like  in  the  camera 
It  was  difficult  to  learn  how  to  use  the  robot  due  to  the  latency.  1 

Longer  course  with  a  greater  various  terrain  types.  1 

The  boundaries  were  hard  to  see  at  some  point,  and  I  was  not  sure  which  1 
direction  to  take. 

The  course  is  difficult  to  navigate  in  the  morning  when  there  is  moisture  on  the  1 


grass.  The  light  reflecting  off  the  grass  makes  it  hard  to  see  the  engineer  tape 
on  the  ground.  The  screen  on  the  display  whites  out  as  you  are  coming  out  of 


the  tunnel  which  makes  you  have  to  slow  or  stop  until  it  balances  out.  The 
HMD  needs  to  be  longer  or  larger  all  around  to  make  it  easier  to  focus  your 
eye  on  the  display  when  you  are  moving.  It  also  needs  something  to  block 
sun  light  from  shinning  into  your  eye. 

The  drawback  to  the  training  was  the  "glitches"  in  the  display  or  the  frame  1 
rates. 

The  HHD  bound  was  easy  because  you  stood  there  and  moved  the  robot.  1 
Both  HMD  and  HHD  were  very  easy  to  work  with;  it  will  be  hard  to  pick  one  as  1 
the  best. 

This  will  be  very  useful  for  clearing  procedures  if  the  HMD  were  more  user  1 
friendly. 

The  HHD  is  just  too  bulky  and  it  would  be  difficult  to  fight  with  both  hands  on  1 
the  monitor. 

While  using  the  dominant  eye  is  fine  in  a  purely  training  environment,  soldiers  1 
need  to  use  the  robot  with  their  non-firing  eye  to  facilitate  robot  operation 
while  carrying  a  firearm 
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Appendix  C.  Post  Iteration 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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APPENDIX  C 


POST  ITERATION 
SAMPLE  SIZE  =  36 


1 .  Using  the  scale  below,  please  rate  your  ability  to  perform  each  of  the  following  tasks 
based  on  your  experience  with  the  display  and  technique  that  you  just  used: 

1  2  3  4  5  6  7 

Extremely  difficult  Very  difficult  Difficult  Neutral  Easy  Very  easy  Extremely  easy 


Condition 

MEAN  RESPONSE 

A* 

B* 

C* 

D* 

Move  the  robot  in  the  correct  direction 

5.26 

5.44 

4.83 

4.40 

Avoid  obstacles 

5.15 

5.09 

4.49 

4.09 

Avoid  pot  holes 

4.86 

4.86 

4.47 

3.81 

Assess  down  slopes  for  navigatability 

4.84 

4.68 

4.66 

3.97 

Assess  side  slopes  for  navigatability 

4.68 

4.79 

4.43 

3.89 

Identify  any  other  terrain  features  that  might  have 
an  adverse  effect  on  the  ability  of  the  robot  to 
maneuver  through  the  terrain 

4.79 

4.91 

4.44 

3.97 

Perform  your  IMT 

5.19 

5.04 

4.69 

4.04 

Drive  the  robot  while  in  a  stationary  position 

5.91 

5.58 

5.47 

4.92 

Anticipate  whether  the  ground  clearance  of  the 
vehicle  will  allow  negotiation  of  rugged  terrain 

5.00 

4.94 

4.64 

4.26 

Anticipate  whether  the  turn  radius  of  the  vehicle  will 
allow  a  turn 

4.80 

4.97 

4.75 

4.00 

Identify  if  the  robot  is  on  the  correct  path 

5.09 

5.29 

4.86 

4.25 

Identify  if  you  are  on  the  correct  path 

5.46 

5.33 

4.97 

4.54 

Navigate  far  enough  ahead  to  plan  route  in 
advance 

5.03 

4.76 

4.91 

4.29 

Navigate  well  enough  to  drive  at  slowest  speeds 

5.61 

5.74 

5.47 

5.21 

Navigate  well  enough  to  drive  at  medium  speeds 

5.26 

5.37 

5.27 

4.72 

Navigate  well  enough  to  drive  at  fastest  speeds 

4.60 

4.63 

4.49 

3.77 

Finish  the  course  quickly 

5.21 

5.43 

4.86 

4.26 

Cross  a  danger  area  effectively 

5.10 

5.00 

4.84 

4.35 

Cross  a  danger  area  quickly 

5.10 

5.17 

4.71 

4.33 

Navigate  the  robot  across  a  danger  area 

5.40 

5.13 

4.84 

4.40 

Ability  to  maintain  situation  awareness  with  this 
display  and  technique 

4.89 

4.58 

4.44 

3.58 

Overall  ability  to  perform  the  course  with  this 
display  and  technique 

5.24 

5.25 

4.74 

4.22 

Overall  ability  to  move  tactically  to  the  objective 
with  this  display  and  technique 

5.29 

4.80 

4.74 

3.97 

*A  -  HHD/bounding;  B  -  HHD/continuous;  C  -  HMI 

D/bounding;  D  -  HMD/continuous 

(Key  applies  throughout  document) 
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Comments 


Responses 


A-  HHD/boundinq 

This  was  the  easiest  and  most  rewarding  trial.  1 

Robot  is  very  easy  to  work  with.  1 

Screen  is  clear  and  easy  to  maneuver.  1 

Larger  screen  allows  for  greater  clarity  for  seeing  upcoming  objects.  1 

This  screen  was  much  easier  for  me  to  use  and  focus  on.  Having  both  eyes  1 
on  the  screen  helped  me  to  visualize  my  path  better  and  maintain  my 
orientation. 

Being  stationary  while  moving  the  robot  allows  for  a  greater  speed  being  1 

implemented  to  complete  the  mission. 

Bounding  allowed  for  more  time  to  concentrate  on  maneuvering  the  robot  2 
because  of  being  in  a  stationary  position. 

Bounding  is  much  easier  than  continuous  movement.  More  difficult  to  maintain  1 
situational  awareness  while  both  operator  and  robot  are  continuously  moving. 

Having  the  pack  on  your  back  was  good  and  I  liked  how  it  was  tight  against  1 
your  body  so  that  it  wouldn’t  flop  all  around. 

With  HHD  it  seems  a  bit  less  tactical  for  situational  awareness  outside  the  1 
display  because  we  are  focused  more  on  the  display. 

Poor  quality  of  picture  meant  that  I  identified  few  of  the  objects  placed  on  the  1 
course. 

Coming  out  of  a  shadowed  area  into  bright  light  causes  the  camera  to  white  1 
out  momentarily.  I  had  to  stop  the  robot  until  the  light  balance  adjusted. 

Sunlight  impairs  screen  sight.  1 

The  display  suffers  a  lot  of  glare  from  sunlight,  unless  the  operator  holds  it  1 


directly  up  to  his/her  face  (difficult  when  walking  with  robot,  but  less  so  when 


stationary),  in  which  case  the  picture  clarity  can  become  sharper  than  the 
HMD. 

B-HHD/continuous 

Controller  comfort  aside,  was  very  easy  to  use.  2 

Good  clarity.  2 

It  was  easier  to  maneuver  today  compared  to  yesterday.  There  was  minimal  1 
lag  today  compared  to  yesterday. 

It  is  ok,  however  would  be  nice  to  have  a  remote  that  is  geared  for  someone  1 
who's  right  handed. 

Fast  paced,  low  visibility  depending  on  terrain.  1 

While  the  bounding  technique  is  more  comfortable  for  me  with  either  device,  I  1 
prefer  the  HHD.  I  performed  better  with  both  bounding  and  continuous 


movement  using  this  device.  For  soldiers  it  is  important  to  maintain  situational 
awareness  at  all  times  and  the  handheld  gives  us  more  mobility  to  assess  our 
surroundings  while  driving  the  robot  at  the  same  time. 
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Comments 


Responses 

It  becomes  disorienting  maneuvering  the  robot  on  a  different  plane  as  me  1 
because  of  peripheral  and  video  feed  views. 

Was  able  to  negotiate  the  course  with  the  robot,  but  meandered  off  the  path  1 
on  foot  and  had  difficulty  focusing  on  anything  but  following  the  robot,  and 
had  I  not  had  the  guide,  I  would  have  veered  off  course  while  guiding  my 
robot. 

Had  to  keep  both  eyes  on  the  screen  so  I  couldn't  see  what  was  going  on  1 
around  me. 

Situational  awareness  minimal  pace  man  a  must  to  navigate  course  while  1 
maintaining  consistent  speed. 

Having  most  difficulty  identifying  "IED"  type  objects  from  the  display.  1 

Once  again,  glare  from  sunlight  severely  impeded  the  effectiveness  of  the  1 
display  --  more  so  than  the  HMD  device. 

With  sun  light  it  is  very  hard  to  see  the  screen,  Also  the  control  is  very  big  and  1 
hard  to  handle. 

Driving  controls  should  be  allowed  to  switch  between  left  and  right  handed  1 
operators. 

It  was  hard  because  the  picture  on  the  hand  mount  would  lag  1 

C-  HMD/bounding 

Easier  to  manipulate  robot  while  stationary,  i.e.,  bounding  technique.  1 

Easier  to  use  HMD  when  bounding  because  I  could  rest  and  focus  my  vision  1 
on  the  display,  and  concentrate  on  moving  the  robot. 

Bounding  worked  very  well  for  me.  I  experienced  very  little  loss  of  clarity  from  1 
sunlight;  bounding,  then  controlling  the  robot  while  stationary  proved 
effective. 

Much  easier  to  move  the  robot  while  you  are  stationary.  Bounding  makes  it  1 
easier  to  observe  your  own  surroundings  while  moving,  as  well  as  makes  it 
easier  to  effectively  control  and  make  observations  while  you  are  moving  the 
robot. 

Task  was  easy  due  to  the  ability  to  switch  eyes  when  switching  tasks.  1 

I  found  this  easier  than  I  thought,  but  I  ended  up  having  to  hold  my  eyepiece  1 
with  my  right  and  use  the  joystick  with  my  left.  This  may  have  been  due  to  the 
fact  that  I  needed  to  adjust  it  more,  but  I  think  it  did  affect  my  overall 
performance. 

Every  time  that  I  stopped,  I  had  to  readjust  the  display  on  my  head.  While  1 
moving  I  had  to  hold  the  display  with  my  right  hand  so  I  would  be  steady  and 
close  enough  to  my  eye  to  avoid  eye  strain.  Also  in  order  to  move  the  robot 
quickly,  I  had  to  close  my  left  eye  to  avoid  losing  focus  and  depth  perception 
in  my  right  eye. 

I  found  it  hard  to  always  see  the  white  engineer  tape.  1 

I  had  to  keep  one  eye  closed  at  all  times  otherwise  I  couldn’t  see  the  path  1 
correctly. 
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Comments 


Level  of  difficulty  was  primarily  experienced  as  I  had  to  navigate  w/o  my  1 

glasses  as  the  head  gear  was  uncooperative. 

Not  able  to  see  when  you  are  in  the  dark  going  into  the  light. 

Screen  needs  to  be  bigger  than  an  inch  cordless  screen  mount.  Needs  to  be  1 
in  night  vision. 

Head  mount  needs  to  be  strapped  better  to  your  head.  1 

It  was  very  doable;  but  there  is  much  room  for  improvement.  The  delay  was  1 

bad,  but  doable. 

There  was  a  good  amount  of  lag  when  the  course  started  but  got  better  as  I  1 
progressed. 

D  -  HMD/continuous 

After  driving  four  times,  getting  the  feel  of  the  remote  is  easier  plus  this  time  1 
there  was  no  lag  in  the  connection  between  receiving  pictures  or  giving 
directions  which  made  maneuvering  the  course  easier. 

It  was  difficult  to  multi-task  while  keeping  one  eye  closed  to  view  the  display,  1 
but  keeping  both  eyes  open  was  more  difficult. 

From  this  display  I  wanted  to  close  the  non-dominate  eye  to  more  closely  1 


watch  what  I  was  doing  with  the  robot.  I  believe  the  helmet  mounted  display 
is  best...  However  in  a  perfect  world,  the  "robot"  view  would  be  superimposed 
over  my  regular  view  to  allow  me  to  retain  all  of  my  peripheral  vision.  Also, 
the  camera  lens  limits  the  peripheral  "vision"  of  the  robot. 

It  is  difficult  to  focus  on  the  head  display  and  still  have  peripheral.  My  eyes  1 
were  severely  competing  with  each  other,  but  it  may  be  due  to  the  fact  that 
the  headgear  was  a  prototype  and  was  fitting  completely  properly. 

Wearing  glasses  and  having  poor  vision,  I  was  unable  to  get  the  display  over  1 

my  glasses  and  had  to  navigate  without  them.  Was  able  to  see  the  screen 
fine,  but  couldn't  see  where  I  was  going  myself  very  well.  Also  feel  the  control 
of  the  robot  would  be  vastly  increased  by  giving  independent  control  over 
each  track  for  tight  turns,  and  better  control. 

The  HMD  fogs  up  when  you  start  to  sweat  because  of  the  heat.  The  screen  1 
could  be  just  a  little  bit  larger  so  your  eyes  are  not  straining  to  adjust  and 
focus  on  a  small  screen.  If  the  eye  piece  could  be  moved  closer  or  the 
peripheral  vision  of  the  eye  that  you  are  using  could  be  minimized  by  using  a 
larger  cover  it  would  help  reduce  eye  fatigue. 

I  had  a  lot  of  difficulty  with  this  movement.  When  I  did  the  bounding  exercise  1 
with  the  HMD  I  felt  really  comfortable.  I  was  able  to  close  one  eye  while  I 
focused  on  the  robots  movement.  With  continuous  movement  and  having  to 
keep  both  eyes  open,  I  felt  extremely  disoriented  and  unable  to  perform  the 
task  comfortably. 

Better  than  the  handheld,  but  still  distracting  and  difficult  to  move  personally  1 
while  operating  the  robot. 

It  was  difficult  to  keep  my  navigation  on  the  robot,  while  trying  to  navigate  my  1 
walk. 
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Comments 


Responses 


Very  difficult  to  move  constantly  and  while  looking  for  threats  and  keeping  the  1 
robot  on  course. 

The  glare  on  the  display  made  it  hard  to  see  sometimes.  1 

The  head  display  cannot  stay  stationary  while  moving.  1 

Tunnel  still  bright;  when  you  come  out  you  can’t  see  very  well.  1 

The  display  feed  from  the  camera  makes  it  difficult  to  identify  objects  and  1 

obstacles  in  your  path  until  the  robot  is  right  on  top  of  it.  The  camera  angle 
makes  it  difficult  to  recognize  and  navigate  slopes,  particularly  descents. 

It  was  kind  of  hard  to  work  with  the  delay  of  the  iRobot.  1 

I  find  that  the  HMD  is  more  difficult  because  it  has  to  be  held  while  operating  1 
with  one  hand. 


2.  Using  the  scale  below,  please  rate  your  ability  to  perform  each  of  the  following 
continuous  movement  tasks  based  on  your  experience  with  the  display  and  technique 
that  you  just  used: 

1  2  3  4  5  6  7 

Extremely  difficult  Very  difficult  Difficult  Neutral  Easy  Very  easy  Extremely  easy 


MEAN  RESPONSE 

HHD 

HMD 

Drive  the  robot  while  you  are  performing  tunnel  maneuver 

5.92 

5.39 

Drive  the  robot  while  you  are  performing  the  low  crawl 

4.25 

5.00 

Drive  the  robot  while  you  are  negotiating  the  hill 

6.00 

5.00 

Drive  the  robot  while  you  are  climbing  the  stairs 

4.67 

5.00 

Drive  the  robot  while  you  are  walking 

5.70 

5.27 

HMD 

It  was  fairly  doable.  1 

When  you’re  looking  through  the  lens,  it’s  not  tight  enough  on  my  head.  1 

HHD 

Found  bounding  to  be  better  for  control  of  the  robot,  as  well  as  being  able  to  1 
observer  your  surroundings. 
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3.  Please  check  any  of  the  following  conditions  that  you  may  have  experienced  during 
this  trial. 


Condition 

NUMBER  OF  RESPONSES 

A 

B 

c 

D 

Eyestrain 

3 

1 

10 

13 

Tunnel  vision 

3 

4 

2 

4 

Headaches 

0 

0 

0 

0 

Motion  sickness 

0 

0 

0 

0 

Nausea 

0 

0 

0 

0 

Disorientation 

3 

4 

7 

7 

Dizziness 

0 

0 

0 

0 

Competition  between  eyes  for  vision  of  the  different 
scenes  at  which  they  are  looking 

0 

2 

8 

10 

Other 

1 

0 

1 

4 

Comments  Responses 


A  -  HHD/boundinq 

Became  winded  from  bounding,  but  it  did  not  impact  my  ability  to  drive  the  1 
robot  significantly.  Sweat  and  heat  were  higher  in  the  bounding  exercise,  but 
also  didn’t  play  a  major  factor  in  driving  the  robot. 

B  -  HHD/continuous 

It  is  difficult  to  change  focus  between  robot  and  my  walking  path.  1 

C  -  HMD/bounding 

Became  very  blurry.  I  couldn't  tell  if  it  was  me  or  the  equipment.  1 

Eyestrain  only  when  using  the  helmet  mounted  display.  It  is  difficult  to  take  a  1 
look  around  for  yourself  and  look  for  the  robot  at  the  same  time. 

Felt  like  my  body  was  still  moving  when  I  look  through  the  lens.  1 

Getting  into  bright  areas  feels  like  I’m  getting  hit  by  a  flash  bang  grenade.  1 

Having  goggles  that  fit.  1 

The  screen  kept  moving  so  I  had  to  adjust  the  screen.  1 

D  -  HMD/continuous 

Head  display  is  easier  to  carry,  but  lacks  stability.  1 

It  is  easy  to  become  focused  on  the  actions  of  the  robot  through  the  camera.  1 

Blurred  vision  in  the  eyes  with  the  HMD  when  sun  light  was  directly  in  front.  1 

It  was  only  because  I  was  trying  to  look  at  the  display  in  my  screen  and  the  1 

squad  leader  who  was  in  front  of  me. 

Keeping  the  eye  piece  steady  over  my  eye.  1 

More  practical  use  and  training  with  this  gear  consistently  will  help  with  eye  1 
adjustments. 

Not  able  to  wear  glasses.  1 

Hard  to  focus  walking  and  driving  robot.  1 

Sweat.  Severe  eye  strain  because  of  having  to  refocus  to  move  and  to  drive  1 
the  robot.  This  led  to  disorientation. 
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4.  Using  the  scale  below,  please  rate  the  following  characteristics  of  the  display  that  you 
just  used: 


1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN  RESPONSE 

HHD 

HMD 

Resolution  (clarity)  of  the  display 

4.97 

4.74 

Size  of  objects  appearing  in  the  display 

5.01 

4.54 

Ability  to  adjust  display 

4.44 

3.87 

Comfort  of  viewing  the  display 

4.96 

3.96 

Display  brightness 

4.94 

4.75 

Effect  of  glare  on  the  display 

3.59 

4.13 

Contrast  between  objects  on  driving  display 

4.77 

4.49 

Display  color 

4.92 

4.87 

Comfort  of  wearing/carrying  the  display 

5.01 

4.33 

Impact  of  display  on  speed  of  movement 

4.76 

4.39 

Display  fogging 

5.10 

4.78 

The  effect  of  the  size  and  shape  of  the  display  on  mobility  of 
the  dismounted  Soldier 

4.97 

4.79 

Adequacy  of  a  display  of  this  type  for  teleoperating  a  robotic 
vehicle 

5.04 

4.66 

Impact  of  display  on  field  of  view  of  the  unaided  eye 

NA 

4.18 

Impact  of  display  on  clarity  of  vision  from  the  unaided  eye 

NA 

4.29 

Comments  Responses 

HHD 

The  joystick  maneuvering  was  in  sync  with  the  movement  of  the  robot,  with  1 
minimal  delay,  so  it  was  easier  to  complete  the  task  today. 

The  screen  should  be  changed  to  a  type  that  isn’t  affected  so  badly  by  1 

sunlight  and  could  also  be  seen  while  being  held  in  direct  sunlight. 

The  HHD  should  be  much  smaller  and  easier  to  hold,  as  well  as  usability  for  1 
controlling. 

Display  should  be  able  to  tilt  to  a  45  degree  angle  to  allow  soldier  to  see  1 

where  he  is  walking  while  looking  at  the  display.  It  would  also  help  reduce 
sun  glare  on  the  screen. 

Would  like  to  be  able  to  adjust  light  settings  to  see  from  a  covered  location.  1 

Glare  impacted  maintaining  positive  position  which  slowed  and  or  stopped  1 

forward  movement. 
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Comments 


Responses 


Display  contrast,  clarity,  and  brightness  are  consistently  the  most  hindering  1 
characteristic  of  the  robotic  device.  Maintaining  situational  awareness  is  very 
difficult,  as  well. 

This  task  was  made  difficult  due  to  equipment  malfunction.  The  display  video  1 
was  slow  with  delays.  After  the  correction,  the  task  was  doable.  Delays  were 
horrific. 

There  were  several  delays  of  at  least  3  seconds  or  more  from  the  actual  robot  1 
movement  and  the  display. 

Camera  had  significant  delay  in  refocusing  from  a  dark  tunnel,  to  a  brightly  lit  1 
area,  causing  movement  to  temporarily  halt. 

HMD 

HMD  is  comfortable  and  effective.  1 

I  really  like  the  handheld;  it  has  a  bigger  display  and  makes  it  easier  to  see  1 
what  is  around. 

The  model  we  were  using  to  test  was  not  mounted  very  well  to  our  heads  and  1 
we  had  to  hold  it  into  place.  Other  than  that  it  was  a  very  good  piece  of 
equipment  to  use. 

This  display  is  small;  however  it  does  have  a  good  picture.  1 

I  think  that  this  display  is  small  and  does  not  sit  well  on  the  person  wearing  1 

them. 

The  clarity  of  objects  viewed  in  the  display  is  relatively  poor.  I  had  no  issues  1 
navigating  the  course  [effectively],  but  I  did  have  problems  identifying 
obstacles  and  foreign  objects. 

Fairly  uncomfortable  especially  for  anyone  who  wears  glasses.  1 

I  answered  neutral  for  HMD  as  I  had  to  wear  my  glasses  during  the  course.  1 
As  a  result,  the  head  gear  and  eye  piece  weren't  properly  aligned. 

I  couldn't  use  my  glasses  during  this  phase  as  the  head  gear  was  1 

uncooperative. 

The  goggles  are  uncomfortable  and  the  display  should  be  a  bit  closer  to  the  1 
eye. 

The  screen  needs  to  be  adjustable  so  that  is  stays  where  you  put  it  when  1 

you're  stationary.  I  had  to  keep  holding  it  close  to  my  eye  just  so  I  could  see 
clearly  where  I  was  going.  Once  I  could  see  clearly  and  not  lose  focus  in  my 
eye,  I  was  able  to  drive  and  react  faster. 

Kept  losing  focus  on  display.  Also  had  to  keep  re-adjusting  display.  1 

Picture  and  clarity  of  robot  perspective  not  as  good  as  could  be.  The  field  of  1 
view  needs  to  be  widened  to  account  for  the  tunnel  vision  of  the  user, 
especially  if  the  user  is  in  view  of  the  robot. 

Experienced  poor  video  clarity  change  when  moving  from  darkness  to  light.  1 
The  limited  vision  from  the  camera  scope  made  the  task  difficult.  The  best  way  1 
to  improve  this  task  is  to  fix  the  delay  and  provide  a  larger  scope  from  the 
video  camera. 
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5.  Using  the  scale  below,  what  is  your  overall  rating  of  the  display  and  technique 
combination  that  you  used  this  iteration? 


1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN  RESPONSE 

A 

B 

C 

D 

5.35 

5.37 

5.06 

4.49 

Comments  Responses 

A  -  HHD/boundinq 

Very  easy  to  move  with  this  device,  easy  to  spot  lEDs  with  this  display,  much  2 
better  display  than  the  head  mounted  display. 

Driving  the  robot  was  easier  using  the  bounding  technique  because  of  the  1 
time  required  to  concentrate  on  using  the  controls  and  deciphering  the 
footage. 

Handheld  was  easy  to  use  in  bounding,  but  I  found  the  HMD  more  1 

comfortable  for  continuous  movement 

I  prefer  the  handheld  over  the  mounted  piece;  it  was  easier  to  see  things  and  1 
the  feed  was  faster  as  well  as  the  reaction  to  when  I  was  giving  directions. 

Display  should  be  able  to  tilt  up  and  down  to  compensate  for  the  glare  on  the  1 
screen  from  direct  sun  light. 

The  experience  was  problematic  due  to  slow  video  malfunction.  After  1 

correction,  the  task  was  doable.  Delays  and  limited  video  focus  display  made 
the  task  a  bit  difficult.  You  fix  these  areas  and  it  would  be  better  in 
accomplishing  this  task. 

The  handheld  control  is  a  little  awkward,  and  camera  to  screen  time  was  a  1 
little  slow. 

B  -  HHD/continuous 

I  think  this  works  very  well  and  I  think  that  with  the  size  of  the  handheld  1 

display  makes  it  easier  and  clearer  to  see  what  is  happening  or  what  is  in 
front  of  you. 

The  equipment  was  sharp  today  with  minimal  delay  between  joystick  controls  1 
and  the  robot,  so  it  was  easier  to  maneuver  the  robot  and  complete  the 
assigned  task. 

Difficult  to  use  HHD  while  walking,  using  peripheral,  and  keeping  aware  of  1 
robot  location. 

Glare  from  the  sun  seems  to  affect  the  HHD  more  than  the  HMD.  The  1 

operator  instinctively  holds  the  HHD  down  in  front  of  them,  and  sunlight  spills 
down  into  the  display  from  above  as  a  result.  This  problem  is  largely  absent 
with  the  HMD. 
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Comments 


Responses 

I  would  really  like  to  see  a  different  type  of  unit  used  for  a  control  once  this  1 
thing  is  used  by  the  soldiers.  The  one  being  used  now  is  way  too  big  and 
cannot  be  seen  in  sunlight,  which  is  often  times  where  it  will  need  to  be  seen. 

C  -  HMD/boundinq 

Overall  I  think  that  the  handheld  has  the  best  display.  1 

Head  display  allows  greater  situational  awareness,  especially  when  actively  1 
operating  the  robot,  and  the  head  display  is  less  cumbersome  and  easier  to 
handle. 

Loved  it.  1 

It  was  good,  but  it  can  be  improved.  1 

Bounding  is  probably  the  best  way  to  utilize  this  device  with  a  dismounted  1 
soldier. 

The  HMD  provides  an  effective  view  using  the  bounding  method;  it  would  be  1 
more  efficient  if  one  didn't  have  to  use  their  free  hand  to  adjust  the  view. 

HMD  +  either  technique  =  success.  1 

Screen  needs  to  be  bigger  and  you  need  to  be  able  to  block  sun  light  from  1 
shinning  in  your  eye  from  over  the  top  of  the  display. 

The  head  display  needs  to  be  a  little  more  secure.  1 

D  -  HMD/continuous 

This  is  a  good  tool.  This  will  improve  and  the  functional  use  of  this  product  will  1 
grow. 

Very  useful  thing  to  have  around.  1 

The  HMD  has  a  natural  and  intuitive  feel  to  it,  but  I  found  myself  shutting  one  1 
eye  to  improve  the  clarity  of  the  picture,  which  decreased  my  situational 
awareness  somewhat. 

After  doing  all  of  the  different  driving  techniques  I  still  prefer  the  handheld  and  1 
I  prefer  the  bounding.  I  found  that  was  the  easiest  way  to  maneuver  and  see 
clearly. 

Navigating  the  robot  while  walking  was  difficult,  due  to  double  navigation  1 

simultaneously.  Bound  navigation  seemed  easier  to  accomplish  versus 
continuous  movement  navigation. 

The  downside  is  picture  quality  and  clarity.  1 

Did  not  like  using  the  HMD.  It  becomes  burdensome  on  the  eyes,  and  I  could  1 
see  it  leading  to  nausea,  or  vomiting  if  used  for  a  lengthier  period  of  time. 

The  display  needs  to  be  larger  to  reduce  eye  strain.  Or  at  least  longer  to  cover  1 
your  field  of  view  in  the  eye  that  you  are  using. 

The  screen  is  hard  to  see  into.  And  the  delay  time  made  navigating  the  robot  1 
difficult. 

The  robot  has  a  lot  of  work  until  it  is  ready  for  combat.  1 
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Intentionally  left  blank. 
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Appendix  D.  Motion  Sickness  Assessment 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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APPENDIX  D 


MOTION  SICKNESS  ASSESSMENT 
SAMPLE  SIZE  =  36 


1.  Using  the  scale  below,  please  rate  how  accurately  the  following  statements 
describe  your  current  state. 

123456789 
Not  at  all  < - >  Severely 


Condition 

MEAN  R 

ESPONSE 

DISPLAY  HHD 

DISPLAY  HMD 

A 

B 

C 

D 

1  felt  sick  to  my  stomach 

1.07 

1.05 

1.04 

1.04 

1  felt  disoriented 

1.69 

1.59 

1.57 

1.88 

1  felt  faint-like 

1.03 

1.00 

1.04 

1.04 

1  felt  tired/fatigued 

1.37 

1.23 

1.22 

1.08 

1  felt  annoyed/irritated 

1.93 

1.36 

1.39 

1.60 

1  felt  nauseated 

1.07 

1.00 

1.09 

1.04 

1  felt  sweaty 

3.00 

2.27 

2.35 

2.44 

1  felt  hot/warm 

2.93 

2.59 

2.39 

2.48 

1  felt  queasy 

1.03 

1.05 

1.09 

1.08 

1  felt  dizzy 

1.03 

1.00 

1.09 

1.04 

1  felt  lightheaded 

1.10 

1.18 

1.09 

1.12 

1  felt  like  1  was  spinning 

1.03 

1.09 

1.09 

1.04 

1  felt  drowsy 

1.03 

1.19 

1.04 

1.08 

1  felt  as  if  1  may  vomit 

1.03 

1.00 

1.04 

1.04 

1  felt  clammy/cold  sweat 

1.10 

1.00 

1.04 

1.04 

1  felt  uneasy 

1.27 

1.23 

1.13 

1.44 
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Appendix  E.  End  of  Experiment 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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APPENDIX  E 


END  OF  EXPERIMENT 
SAMPLE  SIZE  =  36 

1 .  Please  rank  order  the  display  and  technique  in  the  order  of  your  preference  -  with 
1  being  your  favorite,  2  your  second  choice,  etc.  (The  following  are  the  numbers  of 
times  a  display  and  technique  combination  was  chosen  over  the  other  three  possible 
combinations.) 


Movement  and  technique  combination 

Number  of  time  chosen  as  preferred 

HHD  and  bounding  movement 

86 

HHD  and  continuous  movement 

61 

HMD  and  bounding  movement 

43 

HMD  and  continuous  movement 

20 

Comments  Responses 


A-HHD  and  Bounding  movement 

This  was  the  best  and  most  comfortable  technique  to  maneuver  the  robot.  1 
Clarity  of  view  was  much  better.  1 

Easiest  of  all  4  combinations.  3 

Allows  you  to  find  cover,  scout  the  area  you  will  be  moving  through,  and  1 


allows  to  focus  almost  entirely  on  the  robot  making  it  easier  to  maneuver. 

As  a  person  wearing  glasses,  was  the  easiest  for  me.  Was  able  to  see  1 

surroundings  while  navigating  the  robot  at  the  same  time. 

I  liked  it  a  lot.  1 

I  prefer  HHD  and  this  technique  the  most.  1 

It  is  better  because  there  isn’t  a  glare  and  the  picture  looks  really  good  and  1 
being  in  a  steady  position  is  pretty  simple. 

That  was  easier  because  you  are  still  when  you  are  moving  the  robot.  1 

The  best  combination.  It  was  very  easy  to  move  both  myself  and  the  robot  1 

without  making  mistakes. 

The  best  in  my  opinion.  While  maintaining  a  steady  position  during  the  1 

bounding,  I  was  able  to  focus  my  energy  on  driving  the  robot. 

This  was  the  best  technique  because  the  screen  was  larger  which  allowed  the  1 
user  to  spot  lEDs  and  it  was  easier  to  bound  while  carrying  the  controls  for 
the  robot  versus  moving  with  the  controls  on  my  head. 

This  was  my  second  task.  I  found  it  easy  to  drive  the  robot  using  this  display.  1 
It  was  easier  for  me  to  see  what  was  going  on  the  display.  I  found  this  one 
easier  to  use  and  to  see  what  is  going  on 
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Comments 


Responses 


This  technique  is  good,  should  also  be  geared  toward  a  right  handed  person  1 
as  well, 

Very  easy  to  use,  but  impractical  when  it  comes  to  moving  and  fighting  in  1 

combat. 

Almost  as  good  as  HMD  and  continuous  movement,  except  hands  were  not  1 
free. 

Brightness  was  the  only  problem.  1 

Good  picture  on  the  screen;  easier  to  see  what  is  ahead  of  you.  1 

The  bounding  was  easier  than  constant  movement,  but  the  HHD  required  both  1 
hands  and  both  eyes  to  use,  and  eliminated  any  personal  situational 
awareness. 

Hands  are  now  obligated  to  perform  robot  controls  and  weapon  is  maybe  an  1 
afterthought. 

It  was  hard  when  you  had  to  bound  and  let  the  robot  catch  up  with  me.  1 

Lag  was  only  problem.  1 

Only  thing  that  bothers  me  is  that  you  can’t  carry  a  rifle  at  the  same  time.  1 

The  screen  had  to  be  adjusted  to  a  new  angle  so  the  glare  wouldn't  show  on  1 
the  screen. 

If  there  could  be  a  way  to  diminish  the  glare  in  very  sunny  conditions  that  1 

would  be  beneficial. 

I  did  find  that  the  backpack  was  slightly  heavy  and  bulky.  1 

The  stuff  in  there  sounded  loose  and  that  it  may  get  damaged.  1 

B-HHD  and  Continuous  movement 

The  best,  most  preferred  by  me,  easier  to  navigate.  1 

It’s  easy!  1 

I  like  this,  and  is  easiest  with  HHD.  1 

Was  easier  to  move  with  the  robot  and  less  of  a  strain  on  the  eyes.  1 

Favorite  movement  technique.  1 

A  lot  easier  than  HMD.  1 

Besides  walking  while  controlling  the  robot,  was  able  to  control  and  navigate  1 
with  little  problem. 

I  was  more  aware  of  my  surroundings  than  when  using  the  HMD.  It  wasn’t  1 
difficult  to  walk  and  operate  the  robot. 

It  not  that  difficult  because  you  are  walking  when  you  operate  the  robot.  1 

It  was  easier  to  use  the  HHD  while  on  the  move  because  I  can  totally  shift  1 
focus  of  both  eyes  and  use  my  peripheral  to  move.  The  HMD  would  not  allow 
this. 
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Comments 


Responses 


This  technique  was  better  than  the  HMD  continuous  movement,  because  you  1 
could  focus  easier  on  the  team  leader  and  controlling  the  robot  at  the  same 
time. 

This  was  a  good  technique  to  use  if  HHD  bounding  movement  was  not  1 

available.  This  was  my  second  preferable  setup  to  accomplish  robot  tasks. 

This  technique  is  good,  should  also  be  geared  toward  a  right  handed  person  1 
as  well.  If  there  could  be  a  way  to  diminish  the  glare  in  very  sunny  conditions 
that  would  be  beneficial. 

Less  disorienting  than  HMD  continuous  movement.  I  was  able  to  navigate  the  1 
course  fairly  successfully,  though  at  a  much  slower  pace. 

It  was  easy  to  move  with  the  robot,  but  it  was  also  hard  not  knowing  what  was  1 
around  the  blocked  area  and  then  you  walk  up  on  an  IED. 

A  moderately  more  difficult  task  to  perform.  1 

Can’t  see  what  is  ahead  of  you  when  you  are  walking  because  you  have  to  1 
look  down  at  the  screen. 

Glare  was  also  an  issue.  And  I  thought  this  iteration  would  make  it  difficult  for  1 
situation  awareness,  looking  down,  instead  of  up/and  around. 

Hands  are  now  obligated  to  perform  robot  controls  and  weapon  is  maybe  an  1 
afterthought. 

Harder  to  see  objects  on  the  lane  while  moving,  harder  to  keep  on  track.  1 

Kind  of  hard  to  navigate.  1 

Once  again,  easy  to  use  but  the  display  is  too  big  when  it  comes  to  combat  1 
tactics. 

This  was  my  first  task  and  was  a  little  less  difficult  than  the  HMD  moving.  I  1 
think  that  walking  and  driving  at  the  same  time  is  a  little  more  difficult  due  to 
wanting  to  make  the  robot  either  stay  in  front  or  behind  me.  I  would  veer  off 
course. 

C-HMD  and  Bounding  movement 

Easy  to  use  because  I  just  switch  eyes  when  I  need  to  switch  concentration.  1 

Easy;  not  hard  to  navigate.  1 

Easier  to  control  the  robot  from  a  stationary  position.  1 

Bounding  was  easier  with  the  HMD  because  hands  were  free,  I  was  able  to  2 
focus  on  the  display  when  I  was  stopped,  and  it  was  a  comfortable  wear. 

The  HMD  allows  greater  situational  awareness  while  using  the  robot,  is  less  1 
bulky,  requiring  only  one  hand  to  use,  and  the  eye  display  is  less  prone  to 
glare.  The  bounding  technique  is  also  much  easier  than  constant  movement. 

Definitely  better  than  HMD  and  continuous  movement.  1 
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Comments 


Responses 


Was  much  easier  to  negotiate  the  course  because  of  being  able  to  bound  to  1 
your  next  point,  kneel,  and  then  drive  the  robot  to  that  point.  It  allowed  the 
user  to  focus  on  the  course  and  be  able  to  look  for  lEDs. 

I  prefer  this  technique  with  HMD  because  you’re  not  moving  while  you’re  1 

driving. 

I  found  that  bounding  and  then  driving  the  robot  easier  to  do.  1 

It  gave  you  enough  time  to  look  around  at  what  was  going  on.  It  was  also  less  1 
stressful. 

A  slightly  more  complicated  task  to  perform.  1 

Bounding  with  the  HMD  wasn’t  too  bad  but  still  hard  to  walk  with  the  display  1 
over  my  eye. 

Continuous  re-adjusting  of  the  monocle  and  blurred  vision  make  it  difficult  to  1 
maneuver  the  robot.  The  halt  does  help  though. 

I  needed  to  steady  the  eye  display  with  one  hand  and  drive  with  the  other,  but  1 
that  may  have  been  because  it  wasn't  properly  adjusted. 

Eye  piece  cannot  go  over  glasses,  which  makes  observation  of  your  1 

surroundings  difficult. 

Hard  to  keep  display  against  eye  makes  it  difficult  to  see  camera  display  and  1 
focus  on  it  too. 

Holding  eyepiece  was  a  problem.  1 

This  was  my  second  one  and  therefore  I  still  had  some  issues  with  the  1 

controls.  Also,  the  controller  should  be  smaller  and  lighter  so  that  you  can 
tether  it  to  you  so  that  you  may  drop  it  and  engage  in  fire  if  needed  without 
taking  time  to  take  it  off  or  damage. 

I  found  that  my  situational  awareness  suffered  a  bit  more  with  the  HMD  than  1 
the  HHD. 

Initial  disorientation  but  worked  it  out.  1 

It  takes  a  lot  of  getting  use  to.  1 

It  was  difficult  to  focus  one  eye  on  the  screen  and  keep  the  other  eye  focused  1 
on  ambience. 

It  was  easier  to  maneuver  throughout  the  course  when  you  had  to  bound  at  a  1 
position. 

It’s  hard  to  see  the  picture  because  it  doesn’t  fit  right.  1 

Much  easier  to  control  robot  as  opposed  to  continuous  movement.  1 

Needs  to  be  a  little  more  stationary.  1 

Stopping  made  it  easier  to  focus  on  what  the  display  was  showing.  1 

The  picture  on  the  screen  was  small  and  hard  to  recognize  what  obstacles  1 
were  in  the  way. 
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This  technique  is  good,  but  should  also  be  geared  toward  a  right  handed  1 

person  as  well.  The  head  mount  for  covering  the  dominant  eye  needs  to 
cover  the  whole  eye  because  you  tend  to  close  the  other  eye  to  see  more 
clearly  into  the  video  camera  since  there  is  still  come  light  that  filters  through. 

This  was  my  last  task  and  was  easier  than  the  others.  It  was  easier  to  1 

maneuver  once  I  was  bounded  and  the  remote  responded  well.  I  did  have  to 
hold  the  eyepiece  with  one  hand  and  maneuver  with  the  other  which  was  a 
little  on  the  challenging  side  and  would  not  be  beneficial  in  a  fight  because 
both  hands  would  be  occupied. 

You  had  to  hold  the  eyepiece.  1 

D-HMD  and  Continuous  movement 

It  was  easier  because  you  are  walking.  1 

It’s  easy  to  see  where  the  robot  is  going.  1 

Quite  easy,  always  had  one  eye  free  to  see  where  you’re  going.  1 

Was  very  difficult  to  move  with  the  robot  and  look  out  both  eyes  at  the  same  1 
time. 

Can't  focus  on  walking  and  driving  both  at  same  time.  4 

It  was  harder  to  focus  on  the  screen  and  also  watch  where  you’re  going  at  the  1 
same  time. 

Did  not  like  this  technique  at  all  because  it  caused  me  to  miss  lEDs  because  I  1 
had  to  focus  on  the  movement  of  my  team  leader  and  try  to  drive  the  robot 
while  looking  for  lEDs. 

Depth  perception  was  a  little  off.  1 

Harder  to  control  with  the  eye  piece  moving  around.  1 

Harder  to  see  objects  on  the  lane  while  moving,  harder  to  keep  on  track.  1 

Holding  eyepiece  was  a  problem.  1 

I  didn't  enjoy  this  one  at  all.  I  couldn't  maintain  my  orientation  having  to  move  1 
and  keep  my  eyes  focused  on  two  separate  things  at  the  same  time. 

I  think  that  the  eye  display  may  be  harder  to  see  because  it  moves.  I  prefer  1 
the  bounding  technique  because  you’re  not  walking  and  trying  to  watch  the 
screen.  I  also  keep  my  other  eye  closed  while  using  the  HMD  method  so 
walking  while  driving  is  hard. 

It  was  difficult  to  focus  one  eye  on  the  screen  and  keep  the  other  eye  focused  1 
on  ambience. 

It  took  longer  to  focus  on  the  display,  and  the  frustration  factor  was  higher  1 
because  I  had  trouble  staying  on  course  when  I  couldn't  clearly  see  the 
display. 

Of  all  four,  it  is  the  most  difficult.  1 
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I  personally  veered  off  course  several  times  and  had  difficulty  with  IMT  while  1 
using  the  robot. 

The  hardest  of  tasks  to  perform.  1 

The  most  difficult  for  me  as  I  was  unable  to  wear  my  glasses,  so  moving  was  1 
difficult  while  trying  to  pay  attention  to  what  the  robot  was  seeing. 

The  process  was  complicated,  and  movement  made  using  the  display  difficult.  1 
This  technique  really  needs  the  whole  dominant  eye  covered;  you  tend  to  walk  1 
into  the  direction  of  the  movement  of  the  equipment  and  you  might  even  risk 
an  accident  since  you  automatically  close  the  other  eye  to  get  a  more  detail 
visual  of  where  the  vehicle  is  going. 

This  was  my  third  task  and  was  the  worst.  During  this  one  the  communication  1 
between  the  remote  and  the  robot  was  not  good  at  all.  There  was  a  large 
delay  in  feed  for  the  camera  as  well  as  when  I  tried  to  give  it  direction.  I  would 
slightly  push  up  and  nothing  would  happen  so  I  pushed  harder  it  took  time 
and  then  the  robot  would  take  off  fast.  Even  pushing  left  or  right  had  the 
same  effect. 

Too  difficult  to  focus  on  both  tasks  and  switching  concentration.  1 

2.  What  suggestions  do  you  have  for  ways  to  increase  the  effectiveness  of  the 
displays? 

HHD 

I  think  the  HHD  was  great.  1 

I  thought  it  was  really  good;  I  would  make  no  changes.  1 

It  was  excellent,  I  liked  it  best.  2 

It  was  great.  1 

Keep  it  the  same.  3 

Liked  the  larger  screen  size  and  that  you  could  see  it  while  wearing  glasses.  1 
A  wider  field  of  view  would  increase  the  accuracy.  1 

As  it  is,  I  don’t  see  a  practical  advantage  to  carrying  the  display  if  in  fact  an  1 
HMD  can  be  fitted  and  mounted  hands  free. 

Better  camera  resolution/display.  1 

It  needs  to  be  less  bulky  and  heavy.  2 

Get  rid  of  it,  it  is  too  bulky,  cumbersome,  distracting  and  heavy,  and  requires  1 
both  hands  and  eyes  to  use. 

Make  the  controller  not  as  awkward  to  hold.  1 

Make  the  controller  smaller.  9 


57 


Comments 


Responses 


Create  a  smaller  platform.  The  display  wouldn’t  need  to  be  so  big  if  it  worked  1 
more  efficiently  with  various  lighting. 

Make  the  unit  usable  with  only  one  hand.  4 

Having  a  better  shielded  display  that  cuts  the  glare  off  of  the  display  1 

regardless  of  the  position  of  the  handheld. 

Reduce  glare.  4 

Find  a  way  to  block  more  sunlight  from  the  HH  screen.  Sunlight  was  a  factor  1 
that  was  kind  of  making  things  more  hard. 

Use  a  display  screen  that  can  tilt  to  reduce  glare  from  sun  light  on  the  screen.  1 

HMD 

Found  the  HMD  view  most  effective.  1 

A  mount  similar  to  our  NVG  mounts  where  the  monocle  remains  steady  and  1 
you  can  focus  out  the  blur. 

Better  stability  for  the  head  display;  constantly  had  to  adjust  view.  1 

Eyepiece  needs  to  be  more  secure.  4 

It  can  be  difficult  to  keep  the  eyepiece  adjusted  properly  while  moving.  1 

Make  the  eyepiece  cover  your  whole  eye.  1 

Proper  fit  and  adjustment.  I  had  to  hold  the  eyepiece  in  place  and  use  the 
controls. 

Create  a  better  fitting  headpiece,  so  the  display  can  fit  snugly  against  the  eye.  2 
I  really  don’t  know  how  to  keep  the  display  from  moving  while  you’re  walking  1 
unless  you  worked  on  the  mount  and  found  a  way  to  tighten  it  up. 

Be  closer  to  the  face  rather  than  as  far  as  it  was,  1 

Increase  the  size  of  the  display  to  eliminate  competition  between  your  eyes  1 

wanting  to  focus  on  where  you  are  walking. 

Lessen  the  effect  of  sunlight  on  the  display,  and  the  focus  factor  for  leaving  a  1 
dimly  lit  area  to  a  brightly  lit  area  should  be  quicker. 

Make  it  so  that  you  can  still  see  out  of  the  glasses  but  also  make  the  glasses  1 
tinted  so  that  the  displays  camera  appears  on  the  tinted  lens. 

Make  the  display  area  bigger.  2 

Stiffer  armature  to  prevent  wobbling,  better  mounting  system  than  the  1 

goggles,  and  a  focusing  feature  to  adjust  diopters  for  soldiers  with  glasses. 

More  glass  friendly.  2 

More  soldier  level  mounting  adjustments  to  set  up  display  for  individual  eyes.  1 
Move  the  display  closer  to  the  eye  or  create  a  clear  HUD  similar  to  aircraft  1 
HUD. 

Not  really  good  for  combat.  Can't  see  the  area  around  you  and  see  what  the  1 
robot  sees. 
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Something  to  improve  the  clarity  and  brightness  of  the  display.  1 

Talk  to  the  Sony  corporation  and  let  them  come  up  with  something  for  a  1 

display  and  control  device. 

The  arm  that  attaches  the  display  to  the  goggles  needs  to  be  more  ridged  so  it  1 
would  stay  in  place  easier. 

The  camera  completely  obstructs  your  vision.  Open  the  area  up  around  the  1 
camera  to  improve  the  user's  peripheral  vision.  This  might  also  improve  the 
user's  ability  to  drive  the  robot. 

The  display  is  less  panoramic.  1 

Better  communications  between  remote  and  robot!  Better  fitting  is  about  the  1 
only  thing  I  can  think  of. 

Better  equipment.  1 

3.  What  suggestions  do  you  have  for  ways  to  increase  the  effectiveness  of  the  movement 
techniques? 

Bounding  /  Bounding  movement 

Bounding  was  the  most  effective.  No  suggestions.  1 

It  was  excellent,  I  liked  it  best.  1 

It  was  great;  I  enjoyed  it.  1 

Nothing.  It  is  already  two  separate  tasks.  1 

Pretty  good  as  it  is,  unless  there  are  malfunctions  and  you  have  to  go  back  to  1 
the  unit. 

Going  shorter  bounding  distances.  1 

It  was  effective,  but  using  the  unit  without  being  able  to  see  it  more,  would  1 
probably  assist  in  training. 

Lag  occurred  more  frequently.  1 

Making  the  gear  that  you  are  wearing  lighter  is  about  the  only  way  to  improve  1 
this  one. 

Mainly  gear  issues  affected  this  technique.  1 

Operate  the  system  as  a  crew  served  weapon  with  both  an  operator  and  an  1 
assistant  to  provide  security  for  operator. 

Continuous  movement 

It  was  a  good  training.  1 

Being  forced  to  look  at  other  thing  while  navigating  the  robot.  Looking  up  and  1 
down  a  lot  would  increase  the  reality  of  using  the  unit. 
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Have  the  robot  beside  or  slightly  in  front  of  you.  You  would  be  able  to  see  the  1 
robot  when  you  aren't  really  using  it,  so  you  wouldn't  have  to  keep  watching 
your  display  to  move  it. 

Movement  with  the  hand  held  display.  1 

Mainly  gear  issues  affected  this  technique.  1 

Not  preferable,  sorry.  1 

Without  a  guide  to  follow  while  focusing  on  using  the  robot,  it  is  easy  to  veer  1 
personally  off  course. 

Walk  at  a  speed  you  would  actually  move  at  rather  than  a  person  setting  your  1 
pace. 

4.  Please  provide  suggestions  on  ways  to  improve  the  driving  course. 

I  thought  everything  looked  good.  1 

It  was  a  good  challenging  course.  2 

Add  obstacles.  2 

Better  marching  system.  1 

Doing  the  course  on  a  not  so  bright  day.  1 

For  beginners,  make  the  course  on  sand,  so  the  terrain  would  be  easier  to  see  1 
and  navigate. 

Sharper  turns,  sand/dirt  areas  and  objects  to  avoid  should  be  added.  1 

Longer  lane  for  driving  to  get  more  comfortable,  smaller  lEDs  to  make  it  1 

harder  or  in  the  ground. 

Make  it  level  or  give  it  a  little  more  hill  for  training.  2 

Make  it  longer.  3 

Different  terrain.  2 

More  elevation  changes.  1 

More  in  depth  as  far  as  how  to  move  in  confined  areas  and  how  to  make  1 

sharper  turns. 

Raise  the  boundaries  a  little  so  it  is  evident  the  path  to  take.  1 

The  engineer  tape  is  more  difficult  to  see  on  the  ground  through  the  camera.  1 
Raise  the  tape  four  or  five  inches  off  the  ground. 

Would  be  useful  to  be  able  to  independently  move  the  tracks  to  make  tighter  1 
turns.  Would  also  be  excellent  to  make  the  cam  adjust  light  settings  so  it  can 
see  form  a  hidden  position. 
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5.  Please  provide  suggestions  on  ways  to  improve  the  driving  lanes  that  you 
negotiated? 


Comments  Responses 

It  was  a  good  challenging  course.  2 

A  greater  emphasis  on  the  identification  of  foreign  objects  in  the  display.  1 

Additional  iterations  would  build  confidence  in  individual  ability  to  successfully  1 
navigate  course  and  help  to  build  speed. 

Cut  tall  grass.  2 

Cut  grass  in  certain  areas  in  the  tunnel.  Raising  boundaries  will  also  solve  the  1 
problem. 

I  think  that  the  lens  should  have  the  camera  inside  the  lens  to  make  it  easier  1 
to  see. 

I  would  see  if  there  is  something  better  to  outline  the  course  with,  sometimes  it  1 
is  difficult  to  see  the  engineer  tape. 

Use  something  that  does  not  reflect  light  as  much  as  engineer  tape  does  1 

when  it  gets  bright.  When  driving  into  sunlight  the  screen  whites  out  and 
makes  it  hard  to  see  the  tape. 

Make  it  harder;  it  was  not  anywhere  near  challenging  enough  or  realistic  1 

enough  for  me. 

More  obstacles.  2 

More  hills,  holes.  1 

Put  more  obstacles  in  the  lane.  Rocks,  sandbags,  or  maybe  the  use  of  stairs.  1 

Would  be  better  to  make  it  longer  with  varying  terrain  types.  1 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


AAEF 

Air  Assault  Expeditionary  Force 

ANOVA 

analysis  of  variance 

ARVs 

Armed  Robotic  Vehicles 

EXFOR 

Experimentation  Force 

FCS 

Future  Combat  Systems 

HHD 

handheld  display 

HMDs 

head-mounted  displays 

IBCT 

Infantry  Brigade  Combat  Team 

IEDs 

improvised  explosive  devices 

ISR 

intelligence,  surveillance,  and  reconnaissance 

MSAQ 

Motion  Sickness  Assessment  Questionnaire 

NASA 

National  Aeronautics  and  Space  Administration 

ocs 

Officer  Candidate  School 

OCUs 

operator  controller  units 

PDA 

Personal  Digital  Assistants 

SA 

situation  awareness 

SUGV 

Small  Unmanned  Ground  Vehicle 

SVGA 

super  video  graphics  array 

TLX 

Task  Load  Index 

TTPs 

tactics,  techniques  and  procedures 
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FORT  GORDON  GA  30905-5233 
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